Image processing device and method

ABSTRACT

The present invention relates to an image processing apparatus and a method, and in particular to an image processing apparatus and a method preferably applicable to conversion of a wide dynamic-range image having a dynamic range of pixel values wider than the normal one to a narrow dynamic-range image having a narrower dynamic range of pixel values, and to enhancement of contrast. In step S 1 , an input wide-DR luminance image of the current frame is converted into a narrow-DR luminance image based on the intermediate information calculated for the previous frame&#39;s wide-DR luminance image. In step S 2 , the stored intermediate information of the previous frame is updated using the calculated intermediate information. In step S 3 , it is determined if there is any succeeding frame. If there is the succeeding frame, the process returns to step S 1  and processes thereafter are repeated. The present invention is applicable to a digital video camera and the like.

TECHNICAL FIELD

The present invention relates to an image processing apparatus and amethod, and in particular to an image processing apparatus and a methodpreferably applicable to conversion of a wide dynamic-range image havinga dynamic range of pixel values wider than the normal one to a narrowdynamic-range image having a narrower dynamic range of pixel values, andto enhancement of contrast.

BACKGROUND ART

Conventionally, solid-state imaging elements such as CCD (Charge CoupledDevice) and CMOS (Complementary Metal-Oxide Semiconductor) have beenwidely used for the imaging instruments such as video cameras and stillcameras; and optical measurement apparatuses such as componentinspection apparatus used in FA (Factory Automation) and opticalmeasurement apparatuses such as electronic fiberscope used in ME(Medical Electronics).

In recent years, there are proposed a large number of techniques forobtaining an image having a wide dynamic-range (referred to as “wide-DRimage”, hereinafter) of pixel values in comparison with that of opticalfilm photograph using these solid-state imaging elements.

On the other hand, display apparatuses for displaying moving image andstill image such as CRT (Cathode Ray Tube) and LCD (Liquid CrystalDisplay), projection apparatuses such as projector, and various printingapparatuses are not yet widened in their supportable dynamic range ofpixel values at present, and have only a limited range of supportableluminance grayscale. Hence the present status is that, even a wide-DEimage should have successfully pictured, there are no apparatusescapable of displaying, projecting or printing the image as it isobtained.

Hence there is a need for a technique (referred to as “grayscalecompression technique”, hereinafter) by which the dynamic range of pixelvalues of the wide-DR image is narrowed, or in other words, theluminance grayscale is compressed, so as to produce an image (referredto as “narrow-DR image”, hereinafter) adapted to the dynamic range ofthe display apparatuses and so forth.

The following paragraphs will explain a conventionally-proposedgrayscale compression technique. The grayscale compression technique cansimply be achieved by re-assigning the grayscale of pixel values of thewide-DR image so as to be suited to grayscale of a narrower dynamicrange supportable by the display apparatuses or the like.

However, as described in the above, a uniform re-assignment of thegrayscale of pixel values of the wide-DR image simply to the narrowdynamic-range only results in a reduced luminance variation of the imageas a whole, and consequently in conversion into a poorly-looking imagewith a degraded contrast. There are conventionally proposed somegrayscale compression techniques capable of suppressing the loss incontrast. Three grayscale compression techniques ever proposed will beexplained below.

A technique which can be exemplified as a first grayscale compressiontechnique relates to an adaptive determination of a grayscaleredistribution rule based on a histogram of luminance of an inputwide-DR image (more specifically, calculation of a grayscale conversioncurve based on a histogram of an input image). The first grayscalecompression technique is on the premise that a principal subject in animage has a large ratio of occupational area, and is to determine agrayscale conversion curve so as to assign the grayscale as much aspossible to a luminance value at around a peak in the histogram, tothereby suppress lowering in the contrast of at least the principalsubject.

It is, however, difficult to obtain a satisfactory result in everycircumference only by an effort based on the grayscale assignment. In anexemplary case where an image has a plurality of principal subjects andhas a background with a uniform luminance and a relatively wide area(e.g., blue sky), the subjects often fail in obtaining a sufficientgrayscale assigned thereto.

A technique which can be exemplified as a second grayscale compressiontechnique relates to an emphasis of high-frequency components of animage either before or after the grayscale conversion. The secondgrayscale compression technique is to estimate a portion of contrastlost (or supposed to be lost) through the grayscale conversion, and tocompensate the lost portion using a high-frequency filter such as forunsharp masking.

The second grayscale compression technique is advantageous in that itdoes not raise a problem dependent on composition of image unlike thefirst grayscale compression technique. The high-frequency filter is,however, causative of overshoot at the contour portion of the subjectand of noise emphasis at the flat portion, and is therefore understoodas being not capable of always ensuring desirable images.

A technique which can be exemplified as a third grayscale compressiontechnique relates to division of a wide-DR image into alow-frequency-component image and a high-frequency-component image,wherein only the low-frequency-component image is subjected to a propergrayscale conversion processing while leaving thehigh-frequency-component image unmodified, and the both finally added toproduce one synthetic image.

Because the high-frequency-component image is left unmodified in thethird grayscale conversion technique, lowering in the contrast due tothe grayscale conversion is successfully avoidable. The third grayscaleconversion technique, however, still suffers from a problem of overshootat the contour portion of the subject, and noise emphasis at the flatportion similarly to the second grayscale conversion technique, so thatthere is also proposed a method of solving the problem by using anon-linear filter (e.g., median filter) in the process of division intothe low-frequency-component image and high-frequency-component image.

Summarizing now the first to third grayscale compression techniquesdescribed in the above, they can be classified into those effecting thegrayscale compression through a relatively local processing usingneighboring pixels (first and second grayscale compression techniques),and that effecting the grayscale compression using an entire portion ora relatively large area of the image (third grayscale compressiontechnique). The former results in an unnatural image having only thehigh-frequency component thereof enhanced, and is far from successful inobtaining effective grayscale compression results. The latter issuccessful in obtaining a more natural image than obtainable by theformer and is said to be more effective in the grayscale compression,because it can adjust also components of relatively low frequencies inparallel to the emphasis of the high-frequency-components.

The latter, however, suffers from a problem in that the process thereforneeds a large-capacity memory mainly for the delay line or frame memory,so that it is not adaptive to hardware construction. For instance, thethird grayscale compression technique needs a spatial filter fordividing a luminance into a plurality of frequency components, whereinit is necessary to incorporate a large amount of delay lines into thecircuit in order to allow installation of a large spatial filter,because a non-artificial, effective grayscale compression is availableonly when a large spatial filter relative to the image is used.

Meanwhile, for an exemplary case where a function for subjecting awide-DR image to the grayscale compression processing is intended to beinstalled on the output section of an imaging apparatus such as digitalvideo camera and digital still camera, there is a large need for thefunction of grayscale compression processing of the digital stillcamera, for example, to be incorporated into a hardware, becausehigh-speed signal processing is necessary in order to output imagesignals while ensuring a predetermined frame rate. Even for a digitalstill camera for photographing still images, for example, there is ademand for high-speed grayscale compression processing, because it isnecessary to output a monitored image to a finder in order to determinea composition of the image.

As described in the above, there is a strong demand for the grayscalecompression technique which requires only a small memory capacity to beconsumed and a light load of calculation, allows easy hardwareconstruction, and ensures a large grayscale compression effect. Thissort of grayscale compression technique has, however, not been proposedyet.

Additional problems, as described below, commonly reside in theabove-described first to third grayscale compression techniques.

A first problem relates to generation of overshoot in the luminance atthe contour portion of the subject in parallel with emphasis of thehigh-frequency components.

To suppress this, it is necessary to use a relatively large-sized (20×20pixels), two-dimensional, non-linear filter. The two-dimensional,non-linear filter of this size expected as being realized on thesoftware basis, however, raises a problem in that cost for thecalculation will grow extremely high, and that expected as beingrealized on the hardware basis raises a problem in that the circuitscale will grow large due to necessity of a large volume of delay lines.

A second problem relates to control of the amount of contrast emphasisof high-frequency components in the high-luminance region andlow-luminance region. The above-described second and third grayscalecompression techniques are common in that the luminance is divided intoa low-frequency component and a high-frequency component, and thegrayscale compression is effected by enhancing the high-frequencycomponent while keeping the low-frequency component relativelysuppressed.

The emphasis of the high-frequency component, however, results inclipping of the luminance at around the maximum luminance and minimumluminance acceptable by a display apparatus or the like, andconsequently in loss of detail of the image, so that the grayscaleconversion could not be said as being appropriate, and this raises aneed for some countermeasure by which the clipping of luminance isavoidable.

Another problem resides in that an excessive emphasis of the contrastresults in an image having an unnaturally enhanced contour portion ofthe subject, even under a condition not causative of clipping of theluminance.

DISCLOSURE OF THE INVENTION

The present invention is conceived in view of the aforementionedsituation, and an object thereof is to realize a grayscale compressiontechnique which requires only a small memory capacity to be consumed anda less load of calculation, allows easy hardware construction, andensures a large grayscale compression effect.

Another object is to make it possible to appropriately enhance thecontrast of image using a smaller capacity of memory, based on a lessamount of calculation, and based on a simple hardware construction.

An image processing apparatus of the present invention is characterizedby comprising a reduced image generation means for generating a reducedimage from an input image; a correction information acquisition meansfor acquiring a correction information of the input image based on thereduced image; and a grayscale conversion means for converting grayscaleof the input image; wherein the grayscale conversion means correctscontrast of the input image using the correction information, as aprocessing to be performed before and/or after the grayscale isconverted.

The image processing apparatus can further include a smoothing means forgenerating a smoothed image having luminance L_(c) of pixels composingthe input image smoothed based on interpolation calculation using pixelscomposing the reduced image, wherein the grayscale conversion means canbe configured so as to generate a contrast-corrected image based onluminance L_(c) of pixels composing the image, luminance L₁ of pixelscomposing the smoothed image, and a predetermined gain value g.

The grayscale conversion means can be configured so as to calculateluminance L_(u) of pixels composing the contrast-corrected imageaccording to the equation below:L _(u) =g·(L _(c) −L ₁)+L ₁

The reduction means can be configured so as to divide the input imageinto a plurality of blocks, to calculate an average value of luminanceof pixels which belong to the individual blocks, and to produce areduced image constructed from pixels in the same number with that ofthe blocks and having the average value as luminance of the pixels.

The smoothing means can be configured so as to pinpoint a position onthe reduced image corresponded to an interpolated position which is aposition of a pixel to be interpolated, and to use pixels which residein the vicinity of the pinpointed position, to thereby calculateluminance L₁ of pixels of the smoothed image.

The smoothing means can also be configured so as to pinpoint a positionon the reduced image corresponded to a position of interpolation whichis a position of a pixel to be interpolated, and to use 4×4 pixels whichreside in the vicinity of the pinpointed position, to thereby calculateluminance L₁ of pixels of the smoothed image based on bicubicinterpolation.

The image processing apparatus of the present invention can furtherinclude a logarithmic conversion means for subjecting luminance L_(c) ofpixels composing the image before input to the smoothing means tologarithmic conversion, and a logarithmic inversion means for subjectingluminance of pixels composing the contrast-corrected image.

The image processing apparatus of the present invention can furtherinclude a smoothing means for generating a smoothed image havingluminance L_(c) of pixels composing the input image smoothed based oninterpolation calculation using pixels composing the reduced image, anda gain value setting means for setting a gain value g used forcorrecting the contrast; wherein the grayscale conversion means can beconfigured so as to generate a contrast-corrected image based onluminance L_(c) of pixels composing the input image, luminance L₁ ofpixels composing the smoothed image, and a predetermined gain value g;and the gain value setting means can be configured so as to set the gainvalue g based on input initial gain value g₀, reference gain value 1,and an attenuation value attn (Th₁, Th₂, L_(c)) calculated using a firstluminance threshold value Th₁, a second luminance threshold value Th₂,and luminance L_(c) of pixels composing the input image.

The image processing apparatus of the present invention can furtherinclude a conversion means for generating a tone-converted image byconverting luminance L of pixels composing the input image based on aconversion function; a smoothing means for generating a smoothed imageby smoothing luminance L_(c) of pixels composing the tone-convertedimage; and a gain value setting means for setting a gain value g usedfor correcting the contrast based on an initial gain value g₀ whichexpresses an inverse 1/γ of a slope γ of the conversion function;wherein the contrast correction means can be configured so as togenerate a contrast-corrected image based on luminance L_(c) of pixelscomposing the tone-converted image, luminance L₁ of pixels composing thesmoothed image, and a gain value g; and the gain value setting means canbe configured so as to set the gain value g based on input initial gainvalue g₀, reference gain value 1, and an attenuation value attn (Th₁,Th₂, L_(c)) calculated using a first luminance threshold value Th₁, asecond luminance threshold value Th₂, and luminance L_(c) of pixelscomposing the tone-converted image.

The gain value setting means can be configured so as to set the gainvalue g according to the equation below:g=1+(g ₀−1)·attn(Th ₁ ,Th ₂ ,L _(c))

The gain value setting means can be configured also so as to calculatethe attenuation value attn(Th₁,Th₂,L_(c)) according to the equationbelow:attn(Th ₁ ,Th ₂ ,L _(c))=|(L _(c) −Th ₁)/(Th ₂ −Th ₁)|(2Th ₁ −Th ₂ ≦L_(c) ≦Th ₂)

-   -   attn(Th₁, Th₂, L_(c))=1        -   (L_(c)<2Th₁−Th₂, Th₂<L_(c))

The grayscale conversion means can be configured so as to calculate theluminance L_(u) of pixels composing the contrast-corrected imageaccording to the equation below:L _(u) =g·(L _(c) −L ₁)+L ₁

The first luminance threshold value Th₁ can be defined as anintermediate gray level, and the second luminance threshold value Th₂can be defined as a maximum white level.

The reduced image generation means can be configured so as to generate areduced image by converting the input image into the tone-convertedimage based on the conversion function and then by reducing a size ofthe tone-converted image, and the correction information acquisitionmeans can be configured so as to acquire correction informationincluding slope of the conversion function, and the grayscale conversionmeans can be configured so as to correct contrast of the tone-convertedimage based on the reduced image and the slope of the conversionfunction.

The image processing apparatus of the present invention can furtherincludes a hold means for holding the reduced image corresponded to aprevious frame's image and a slope of the conversion function applied tothe previous frame's image.

The reduced image generation means can be configured so as to convert apixel value of an image of the current frame by step-wisely using one ormore conversion functions, and the grayscale conversion means can beconfigured so as to generate the contrast-corrected image by correctingcontrast of the tone-corrected image based on the reduced image held bythe hold means and a product of the slopes individually corresponded toone or more conversion functions.

Of one or more conversion functions, at least one conversion functioncan be configured as a monotonously-increasing function.

The image processing apparatus of the present invention can furtherinclude an average value calculation means for calculating an averagevalue of pixel values of the tone-converted image, and of one or moreconversion functions, at least one conversion function can be configuredso as to have a slope in proportion to an inverse of the average valuecalculated by the average value calculation means.

The average value calculation means can be configured so as to dividethe tone-corrected image into a plurality of blocks, and so as tocalculate, as the average value, a value by weighted addition ofaverages of pixel values of the individual blocks.

The reduced image generation means can be configured so as to generate afirst reduced image by reducing a size of the tone-converted image, andto multiply the individual pixel values of the first reduced image by avalue in proportion to an inverse of an average value of pixel values ofthe first reduced image, to thereby generate a second reduced image.

The image processing apparatus of the present invention can furtherinclude a logarithmic conversion means for subjecting pixel values of animage in the current frame to logarithmic conversion, and a logarithmicinversion means for subjecting pixel values of the contrast-correctedimage to the logarithmic inversion.

The image processing apparatus of the present invention can furtherinclude a gamma conversion means for subjecting pixel values of thecontrast-corrected image to gamma conversion; a luminance rangeinformation calculation means for calculating luminance rangeinformation which indicates a distribution range of luminance componentsof the contrast-corrected image after gone through the gamma conversionby the gamma conversion means; and a normalization means fornormalizing, into a predetermined range, the distribution of pixelvalues of the contrast-corrected image after gone through the gammaconversion by the gamma conversion means based on the luminance rangeinformation calculated by the luminance range information calculationmeans.

The luminance range calculation means can be configured so as tocalculate, as the luminance range information, an upper limit value anda lower limit value of the luminance components of thecontrast-corrected image after gone through the gamma conversion by thegamma conversion means, and the normalization means can be configured soas to convert pixel values of the contrast-corrected image so that theupper limit value and the lower limit value of the luminance componentsof the contrast-corrected image calculated by the luminance rangeinformation calculation means respectively coincide with the upper limitvalue and the lower limit value of a range of luminous componentreproducible by an assumed reproduction apparatus.

The hold means can be configured so as to hold previous frame'sluminance range information calculated by the luminance rangeinformation calculation means.

The image can be a monochrome image constructed from pixels havingluminance components.

The image can be a color image constructed from pixels having aplurality of color components.

The reduced image generation means can be configured so as to generate afirst luminance image constructed from pixels having luminancecomponents based on the color image, to convert the first luminanceimage into the tone-converted luminance image, and to generate a colortone-converted image constructed from pixels having a plurality of colorcomponents based on the tone-converted luminance image.

The reduced image generation means can be configured so as to calculatethe individual color components of the tone-converted image, bycalculating difference values between values of the individual colorcomponents of the color image and values of the luminance components,then by calculating a product of the difference values and slope of theconversion function, and by adding the product to the values of theindividual color components of the tone-converted luminance image.

The reduced image generation means can be configured so as to calculatethe individual color components of the tone-converted image, bycalculating an average value of the luminance component of the firstluminance image, then by calculating a coefficient in proportion to aninverse of the average value, and by multiplying values of theindividual color components of the color image by the coefficient.

The grayscale conversion means can be configured so as to generate acolor contrast-corrected image, by generating a second luminance imageconstructed from pixels having luminance components based on the colortone-converted image, then by correcting contrast of the colortone-converted image generated by the conversion means based on thesecond luminance image, the reduced image held by the hold means and theslope of the conversion function, to thereby generate the colorcontrast-corrected image.

The image processing apparatus of the present invention can furtherinclude a gamma conversion means for subjecting pixel values of thecolor contrast-corrected image to gamma conversion; a luminance rangeinformation calculation means for generating a third luminance imageconstructed from pixels having luminance component based on the colorcontrast-corrected image after gone through the gamma conversion by thegamma conversion means, and for calculating luminance range informationwhich indicates a distribution range of luminance components of thethird luminance image; and a normalization means for normalizing, into apredetermined range, the distribution of pixel values of the colorcontrast-corrected image after gone through the gamma conversion by thegamma conversion means based on the luminance range informationcalculated by the luminance range information calculation means.

The image processing method of the present invention is constructed froma reduced image generation step for generating a reduced image from aninput image; a correction information acquisition step for acquiring acorrection information of the input image based on the reduced image;and a grayscale conversion step for converting grayscale of the inputimage; wherein the grayscale conversion step corrects contrast of theinput image using the correction information, as a processing to beperformed before and/or after the grayscale is converted.

According to the image processing apparatus and method of the presentinvention, it is made possible to generate a reduced image from an inputimage, to acquire correction information based on the generated reducedimage, and to convert grayscale of the input image. In the processing ofthe grayscale conversion, contrast of the input image is corrected usingthe correction information, as a processing to be performed beforeand/or after the grayscale is converted.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an exemplary configuration of adigital video camera according to one embodiment of the presentinvention;

FIG. 2 is a block diagram showing a first exemplary configuration of aDSP shown in FIG. 1;

FIG. 3 is a block diagram showing a first exemplary configuration of atone curve correction section shown in FIG. 2;

FIG. 4 is a graph showing an exemplary tone curve;

FIG. 5 is a block diagram showing a second exemplary configuration ofthe tone curve correction section shown in FIG. 2;

FIG. 6 is a block diagram showing a third exemplary configuration of thetone curve correction section shown in FIG. 2;

FIG. 7 is a block diagram showing an exemplary configuration of areduced image generation section shown in FIG. 2;

FIG. 8 is a block diagram showing an exemplary configuration of anaverage value calculation section shown in FIG. 7

FIG. 9 is a block diagram showing an exemplary configuration of acontrast correction section shown in FIG. 2;

FIG. 10 is a block diagram showing an exemplary configuration of aninterpolation section shown in FIG. 9

FIG. 11 is a drawing for explaining processing of the interpolationsection shown in FIG. 9;

FIG. 12 is a block diagram showing an exemplary configuration of a gainvalue setting section shown in FIG. 9;

FIG. 13 is a block diagram showing an exemplary configuration of acontrast emphasizing section shown in FIG. 9;

FIG. 14 is a drawing for explaining processing in a luminance rangenormalization section shown in FIG. 2;

FIG. 15 is a block diagram showing an exemplary configuration of aluminance range information calculation section shown in FIG. 2;

FIG. 16 is a block diagram showing an exemplary configuration of aluminance range normalization section shown in FIG. 2;

FIG. 17 is a block diagram showing an exemplary configuration of acomposite section substitutable to a portion ranging from the tone curvecorrection section to the contrast correction section shown in FIG. 2;

FIG. 18 is a flow chart for explaining grayscale compression processingby the first exemplary configuration of the DSP;

FIG. 19 is a flow chart for explaining details of the processing in stepS1 shown in FIG. 18;

FIG. 20 is a flow chart for explaining details of the processing in stepS2 shown in FIG. 18;

FIG. 21 is a block diagram showing a second exemplary configuration ofthe DSP shown in FIG. 1;

FIG. 22 is a block diagram showing a first exemplary configuration of atone curve correction section shown in FIG. 21;

FIG. 23 is a block diagram showing a second exemplary configuration ofthe tone curve correction section shown in FIG. 21;

FIG. 24 is a block diagram showing a third exemplary configuration ofthe tone curve correction section shown in FIG. 21;

FIG. 25 is a block diagram showing an exemplary configuration of areduced image generation section shown in FIG. 21;

FIG. 26 is a block diagram showing an exemplary configuration of acontrast correction section shown in FIG. 21;

FIG. 27 is a block diagram showing an exemplary configuration of acomposite section substitutable to a portion ranging from the tone curvecorrection section to the contrast correction section shown in FIG. 21;

FIG. 28 is a block diagram showing an exemplary configuration of aluminance range information calculation section shown in FIG. 21;

FIG. 29 is a flow chart for explaining grayscale compression processingby the second exemplary configuration of the DSP;

FIG. 30 is a flow chart for explaining details of the processing in stepS43 shown in FIG. 29;

FIG. 31 is a flow chart for explaining details of the processing in stepS44 shown in FIG. 29;

FIG. 32 is a block diagram showing an exemplary configuration of animage processing system applied with the present invention;

FIG. 33 is a flow chart for explaining operation of an image processingsystem shown in FIG. 32;

FIG. 34 is a block diagram showing a first exemplary configuration ofthe image processing apparatus shown in FIG. 32;

FIG. 35 is a block diagram showing an exemplary configuration of a tonecurve correction section shown in FIG. 34;

FIG. 36 is a drawing showing an exemplary tone curve used in the firstexemplary configuration of the image processing apparatus;

FIG. 37 is a block diagram showing an exemplary configuration of asmoothed luminance generation section shown in FIG. 34;

FIG. 38 is a block diagram showing an exemplary configuration of areduced image generation section shown in FIG. 37;

FIG. 39 is a block diagram showing an exemplary configuration of anaverage value calculation section shown in FIG. 38;

FIG. 40 is a block diagram showing an exemplary configuration of aninterpolation section shown in FIG. 37;

FIG. 41 is a block diagram showing an exemplary configuration of a gainvalue setting section shown in FIG. 34;

FIG. 42 is a block diagram showing an exemplary configuration of acontrast correction section shown in FIG. 34;

FIG. 43 is a flow chart for explaining grayscale-compressed imagegeneration processing by the first exemplary configuration of the imageprocessing apparatus;

FIG. 44 is a block diagram showing a second exemplary configuration ofthe image processing apparatus shown in FIG. 32;

FIG. 45 is a flow chart for explaining grayscale-compressed imagegeneration processing by the second exemplary configuration of the imageprocessing apparatus; and

FIG. 46 is a block diagram showing an exemplary configuration of ageneral-purpose personal computer.

BEST MODES FOR CARRYING OUT THE INVENTION

A digital video camera as one embodiment of the present invention willbe explained below referring to the attached drawings.

FIG. 1 shows an exemplary configuration of a digital video camera as oneembodiment of the present invention. The digital video camera 1 takes apicture of a subject, generates a wide-DR image having a dynamic rangeof pixel values wider than a general one, saves the image into apredetermined storage medium, and outputs the wide-DR image afterconverting it into a narrow-DR image having a dynamic range of pixelvalues narrower than a general one, to a built-in display also used as acomposition-determining finder or as an image monitor, or to externalapparatuses.

The digital video camera 1 is roughly constructed from an opticalsystem, a signal processing system, a recording system, a display systemand a control system.

The optical system is constructed from a lens 2 for condensing aphoto-image of a subject, a stop 3 for regulating light energy of thephoto-image, and a CCD image sensor 4 for generating a wide-DR image byphoto-electric conversion of the condensed photo-image at apredetermined frame rate. It is to be noted that the description belowwill deal with two cases where the wide-DR image generated by the CCDimage sensor 4 is a monochrome image constructed from one-channelluminance signal, and is a color image constructed from multi-channel(e.g., three-channel) luminance signal.

The signal processing system is constructed from a correlated doublesampling circuit (CDS) 5 for reducing noise by sampling the wide-DRimage output from the CCD image sensor 4, an A/D converter 6 foreffecting AD conversion of the wide-DR image removed with noise by thecorrelated double sampling circuit 5 to thereby obtain a value having abit width of, for example, 14 to 16 bits or around, and a DSP (DigitalSignal Processor) 7 for effecting grayscale compression processing ofthe wide-DR image output by the A/D converter 6.

An image signal having a large number of grayscale, just like thewide-DR image output from the A/D converter 6 and having a bit width of14 to 16 bits, cannot fully be reproduced by general video signalsincluding luminance Y and color difference signals Cr, Cb, but thegrayscale thereof is compressed by the grayscale compression processingby the DSP 7 to a range allowing reproduction by the general videosignals including luminance Y and color difference signals Cr, Cb. TheDSP 7 will be detailed referring to FIG. 2 and succeeding drawings.

The recording system of the digital video camera 1 is constructed from aCODEC (Compression/Decompression) 12 which plays a part in encoding thewide-DR image or narrow-DR image received from the DSP 7 and inrecording it into a memory 13, and in reading and encoding the code datastored in the memory 13 and supplying it to the DSP 7, and a memory 13for storing the encoded wide-DR image or narrow-DR image, constructedfrom a magnetic disk, optical disk, magneto-optical disk, semiconductoror the like.

The display system is constructed from a D/A converter 9 which takes apart in DA conversion of the narrow-DR image supplied from the DSP 7, avideo encoder for outputting the analog narrow-DR image output from theD/A converter 9 to a display 11 after converting it into a general videosignals including luminance Y and color-difference signals Cr, Cb, andthe display 11 typically is constructed from an LCD (Liquid CrystalDisplay) and so forth, which functions as a finder or video monitor bydisplaying an image corresponded to the video signals.

The control system is constructed from a timing generator (TG) 8 forcontrolling operation timing of the components from the CCD image sensor4 to the DSP 7, an input apparatus 15 for accepting various operationsby the user, and a CPU (Central Processing Unit) 14 for controlling theentire portion of the digital video camera 1.

Next, outline of the operations of the digital video camera will beexplained. An optical image of a subject (incident light) comes throughthe lens 2 and stop 3 to the CCD image sensor 4, undergoesphoto-electric conversion by the CCD image sensor 4, and the obtainedelectric signals expressing pixels of the wide-DR image is removed withnoise by the correlated double sampling circuit 5, digitized by the A/Dconverter 6, and supplied to the DSP 7.

The DSP 7 takes a part in grayscale compression processing of thewide-DR image received from the A/D converter 6 to thereby generate anarrow-DR image, and outputs it to the D/A converter 9 or the CODEC 12,or to the both. The narrow-DR image supplied to the D/A converter 9 issubjected to DA conversion, and is then converted into normal videosignals by the video encoder 10, and the resultant image is displayed onthe display 11. On the other hand, the narrow-DR image supplied to theCODEC 12 is encoded and recorded in the memory 13.

Here is an end of description on the overall operation of the digitalvideo camera 1.

Next, the DSP 7, which is the key for the present invention, will bedescribed.

FIG. 2 shows a first exemplary configuration of the DSP 7 adapted to thewide-DR image which is a monochrome image. The monochrome wide-DR imageinput to the DSP 7 is referred to as wide-DR luminance image L,hereinafter. Pixel value (i.e., luminance value) of the wide-DRluminance image is expressed as L(p). In this context, p is a vector orcoordinate expressing pixel position on the image, such as p=(x,y). Itis, therefore, determined to use L(p), which contains both informationof the pixel position and luminance value, as being discriminated from Lwhich expresses the wide-DR luminance image. The same will apply also toother images and the pixel values thereof described later.

The DSP 7 is designed so that luminance L(p) of the wide-DR luminanceimage L is input thereto according to the order of raster.

In the first exemplary configuration of the DSP 1, a logarithmicconversion section 21 subjects the input luminance L(p) to logarithmicconversion, and outputs the obtained logarithmic luminance logL(p) to atone curve correction section 22. The tone curve correction section 22applies a preliminarily-obtained tone curve to the input logarithmicluminance logL(p) and converts it in the direction of compressing thegrayscale, and outputs the obtained logarithmic luminance logL_(c)(p) toa reduced image generation section 23 and a contrast correction section25. The tone curve correction section 22 outputs a representative valueγ which expresses a slope of the applied tone curve to the contrastcorrection section 25. The representative value γ which expresses aslope of the applied tone curve will simply be referred to asrepresentative value γ, hereinafter.

The reduced image generation section 23 generates a reduced imagelogL_(cl) based on the logarithmic luminance logL_(c)(p), correspondedto a single frame, received from the tone curve correction section 22,and makes a reduced image memory 24 to store it.

The contrast correction section 25 corrects the contrast, which wasweakened by the tone curve correction, of the logarithmic luminancelogL_(c)(p) of the current frame received from the tone curve correctionsection 22, based on the representative value γ and the previous frame'sreduced image logL_(cl) held in the reduced image memory 24, and outputsthe obtained logarithmic luminance logL_(u)(p) to a logarithmicinversion section 26. The logarithmic inversion section 26 subjects thelogarithmic luminance logL_(u)(p) having the corrected contrast tologarithmic inversion, and outputs the obtained luminance L_(u)(p)expressed by the normal axis to a gamma correction section 27.

The gamma correction section 27 subjects the luminance L_(u)(p) receivedfrom the logarithmic inversion section 26 to gamma correction inconsideration of gamma characteristics of a reproduction apparatus(e.g., display 11), and then outputs the obtained luminance Y(p) afterthe gamma correction to a luminance information calculation section 28and a luminance range normalization section 30. The luminanceinformation calculation section 28 calculates luminance rangeinformation, which indicates a luminance distribution, for eachluminance Y(p) corresponded to a single frame received from the gammacorrection section 27, and allows them to be held by a luminance rangeinformation memory 29. It is to be noted herein that the luminance rangeinformation refers to information indicating a distribution range ofluminance within one frame, by which luminance Y_(d) closest to darknessand luminance Y_(b) closest to brightness are calculated as luminancerange information [Y_(d), Y_(b)].

The luminance range normalization section 30 converts the luminance Y(b)of the current frame received from the gamma correction section 27,based on the previous frame's luminance range information [Y_(d), Y_(b)]held by the luminance range information memory 29, so that thedistribution range thereof coincides with a range which can be expressedby a reproduction apparatus (e.g., display 11), and outputs the obtainedluminance Y_(n)(p) to the succeeding steps as pixel values of thenarrow-DR image.

As has been described in the above, in the process of the grayscalecompression processing according to the first exemplary configuration ofthe DSP 7, the reduced image logL_(cl) is generated by the reduced imagegeneration section 23, and the luminance range information [Y_(d),Y_(b)] is calculated by the luminance range information calculationsection 28. The reduced image logL_(cl) and luminance range information[Y_(d), Y_(b)] are referred to as an intermediate information,hereinafter.

With DSP 7, the intermediate information is calculated for theindividual frames of the input wide-DR image, and the calculatedintermediate information is used for processing the wide-DR image comingone frame after.

Although it is necessary in general to use information calculated basedon the luminance values for the entire portion or a wide range of theimage for the purpose of carrying out an effective grayscalecompression, a problem arises in the mounting in that a time lag beforethe information is calculated will increase. The DSP 7 therefore usesthe previous frame's intermediate information to the grayscalecompression for the current frame, by selecting the informationextremely less likely to vary with time. This configuration makes itpossible to avoid expansion of the memory consumption and circuit scaleeven after the mounting.

Next, details of the first exemplary configuration of the DSP 7 will bedescribed referring to the attached drawings.

FIG. 3 shows a first exemplary configuration of the tone curvecorrection section 22. In the first exemplary configuration, a LUTmemory 41 preliminarily holds a lookup table (referred to as LUT,hereinafter) which corresponds with a monotonously-increasing tone curveas shown in FIG. 4, and a representative value γ which expresses a slopeof the tone curve. It is also allowable that a function equivalent tothe tone curve may be held in place of the LUT. A table referencesection 42 corrects the logarithmic luminance logL(p) into logarithmicluminance logL_(c)(p) based on the LUT held in the LUT memory 41.

FIG. 4 shows an example of the tone curve, wherein the abscissa plotsinput luminance L(p), and the ordinate plots luminance L_(c)(p) aftertone curve correction, respectively on the logarithmic axes normalizedover a range of [0, 1]. Application of the monotonously-increasingmoderate inverse-S-shaped curve as shown in this example will not giveso strong effect of grayscale compression in the high luminance regionand low luminance region, so that it is possible to obtain a desirabletone with less degree of whiteout or blackout even after the grayscalecompression. On the contrary, the grayscale compression will stronglyaffect the middle luminance region, but this means that the contrastcorrection described later can fully be applied with the middleluminance region, and results in a desirable narrow-DR image with a lessdegree of contrast correction also in the middle luminance range.

It is to be noted that the representative value γ expressing slope ofthe tone curve can be determined typically by finding the slope valuesover the entire luminance range and by determining an average of thesevalues as the representative value γ. The tone curve shown in FIG. 4 hasa representative value γ of 0.67.

FIG. 5 shows a second exemplary configuration of the tone curvecorrection section 22. Unlike the first exemplary configuration, thesecond exemplary configuration does not use a preliminarily-obtainedLUT, but calculates a representative value γ for every frame andcorrects the logarithmic luminance logL(p) into logarithmic luminancelogL_(c)(p). In the second exemplary configuration, an average luminancecalculation section 51 calculates an average value μ of the logarithmicluminance logL(p) of one frame. A divider 52 divides a predeterminedconstant logL_(T) by the average value μ, to thereby calculate therepresentative value γ. A γ memory 53 holds the representative valuereceived from the divider 52. A multiplier 54 multiplies the logarithmicluminance logL(p) of the current frame by the previous frame'srepresentative value γ held by the γ memory 53, to thereby calculate thelogarithmic luminance logL_(c)(p) after the tone curve correction.

Assuming now that the predetermined constant logL_(T) is defined as amoderate-level logarithmic luminance, the average value of thelogarithmic luminance logL(p) for one frame is converted into thelogarithmic luminance logL_(c)(p) after the tone curve correction havingthe same value with the logL_(T).

Although the representative value γ is calculated for every frame, thevalue is supposed to differ not largely between the preceding frame andsucceeding frame, because it is practically calculated based on theaverage value μ of the logarithmic luminance logL(p). Hence, also forthe representative value γ is designed to use the one-frame precedencefor the tone curve correction for the current frame, similarly to theabove-described reduced image logL_(cl) and luminance range information[Y_(d), Y_(b)]. It is therefore defined that also the representativevalue γ is included into the intermediate information.

FIG. 6 is a third exemplary configuration of the tone curve correctionsection 22. The third exemplary configuration is, so as to say, acombination of the first exemplary configuration and the secondexemplary configuration. In the third exemplary configuration, a LUTmemory 61 preliminarily holds a LUT, which corresponds with the tonecurve as shown in FIG. 4, and a representative value γ₁ which expressesa slope of the tone curve. A table reference section 62 corrects thelogarithmic luminance logL(p) into logarithmic luminance logL_(c′)(p)based on the LUT held in the LUT memory 61 k, and outputs it to anaverage luminance calculation section 63 and a multiplier 66.

The average luminance calculation section 63 calculates an average valueμ of the logarithmic luminance logL_(c′)(p) for one frame, and outputsit to a divider 64. The divider 64 divides a predetermined constantlogL_(T) by the average value μ, to thereby calculate a representativevalue γ₂, and allow a γ₂ memory 65 to store it. A multiplier 66multiplies the logarithmic luminance logL_(c′)(p) of the current frameby the previous frame's representative value γ₂ held by the γ memory 65,to thereby calculate the logarithmic luminance logL_(c)(p) after thetone curve correction. A multiplier 67 outputs a product of therepresentative values γ₁, γ₂ as a representative value γ(=γ₁·γ₂) to thecontrast correction section 25 in the succeeding stage.

FIG. 7 in the next shows an exemplary configuration of the reduced imagegeneration section 23. A sorting section 71 of the reduced imagegeneration section 23 sorts the logarithmic luminance logL_(c)(p) forone frame received from the tone curve correction section 22 in thepreceding stage according to blocks to which the luminance belongs whenthe entire image is divided into m×n blocks, and then supplied to theaverage value calculating sections 72-1 to 72-N (=m×n). For example,those classified into the first block are supplied to average valuecalculation section 72-1, and those classified into the second block aresupplied to the average value calculation section 72-2. The same willapply also to the succeeding ones, and those classified into the N-thblock are supplied to the average value calculation section 72-N. Thefollowing description adopts a simple notation of average valuecalculation section 72 when there is no need of discrimination of theindividual average value calculation sections 72-1 to 72-N.

The average value calculation section 72-i (i=1,2, . . . ,N) calculatesan average value of the logarithmic luminance logL_(c)(p) classifiedinto the i-th block, out of logarithmic luminance values logL_(c)(p) forone frame, and outputs it to a composition section 73. The compositionsection 73 generates an m×n-pixel reduced image logL_(cl) having, aspixel values, the average values of the logarithmic luminancelogL_(c)(p) respectively received from the average value calculationmeans 72-i, and makes the reduced image memory 24 in the succeedingstage to store it.

FIG. 8 shows an exemplary configuration of the average value calculationsection 72. An adder 81 of the average value calculation section 72 addsa value held by a register (r) 82 to the logarithmic luminancelogL_(c)(p) received from the sorting section 71 in the preceding stage,to thereby update the value held by the register (r) 82. A divider 83divides a value finally held by the register 82 by the number of pixelsQ composing one block, to thereby calculate an average value of Qlogarithmic luminance logL_(c)(p) values classified into one block.

FIG. 9 in the next shows an exemplary configuration of the contrastcorrection section 25. An interpolation position designation section 91acquires a pixel position p of the logarithmic luminance logL_(c)(p)received from the tone curve correction section 22 in the precedingstage (also referred to as position of interpolation p, hereinafter),and outputs it to an interpolation section 92. The interpolation section92 calculates, by interpolation, the pixel logL_(cl) (p) corresponded tothe position of interpolation p, using the previous frame's reducedimage logL_(cl) held by the reduced image memory 24, and outputs it to acontrast enhancement section 94.

A gain value setting section 93 calculates a gain value g (p), whichdetermines the amount of contrast enhancement of the logarithmicluminance logL_(c)(p) of the current frame, based on the previousframe's representative value γ received from the tone curve correctionsection 22 and based on the logarithmic luminance logL_(c) (p) of thecurrent frame. The contrast enhancement section 94 calculates thelogarithmic luminance logL_(u)(p) having an enhanced contrast in thefrequency components other than low-frequency ones, based on thelogarithmic luminance logL_(c)(p) of the current frame, the gain valueg(p) and the interpolation value logL_(cl) (p) of the reduced image.

FIG. 10 shows an exemplary configuration of the interpolation section92. The interpolation section 92 interpolates the pixel logL_(cl) (p)corresponded to the position of interpolation p based on bicubicinterpolation using 4×4 pixels in the vicinity of the position ofinterpolation p of the previous frame's reduced image logL_(cl).

A vicinity selection section 101 acquires, upon reception of theposition of interpolation p, 4×4-pixel pixel value a[4] [4] in thevicinity of the position of interpolation p, based on the previousframe's m×n-pixel reduced image logL_(cl) held by the reduced imagememory 24, and outputs it to the products summation section 104. Anotation of a[i][j] herein means that pixel value a is an i×jtwo-dimensional arrangement data. The vicinity selection section 101outputs horizontal displacement dx and vertical displacement dy betweenthe acquired pixel value a[4] [4] and position of interpolation p to ahorizontal coefficient calculation section 102 or a vertical coefficientcalculation section 103, respectively.

Relations of the position of interpolation p, neighboring pixel valuea[4][4] and displacements dx, dy herein will be described referring toFIG. 11.

An m×n grid shown in FIG. 11 expresses an m×n-pixel reduced imagelogL_(cl). Assuming now that the position of interpolation p=(px, py) isgiven, a position q on the reduced image logL_(cl) corresponded to theposition of interpolation p will be given as q=(qx, qy)=(px/bx−0.5,py/by−0.5), where (bx, by)=(the number of horizontal pixel of imagelogL_(c)/m, the number of vertical pixel of image/n).

To acquire neighboring pixels around the position q on the reduced imagecorresponded to the position of interpolation p, it is recommended toacquire pixels of the reduced image logL_(cl) which fall within a rangeof qx−2<x<qx+2 and qy−2<y<qy+2 as indicated by hatching in FIG. 11. Inthe area indicated by the hatching, 4×4 positions marked with “+” arepositions of the pixels to be acquired. Displacement (dx, dy) betweenthe neighboring pixel and position of interpolation p is defined as adifference with respect to a nearest pixel in the left downwarddirection. That is, displacement can be given as (dx, dy)=(decimalportion of qx, decimal portion of qy).

Now referring back to FIG. 10, the horizontal coefficient calculationsection 102 calculates a horizontal tertiary interpolation coefficientk_(x)[4] based on the horizontal displacement dx received from thevicinity selection section 101. Similarly, the vertical coefficientcalculation section 103 calculates a vertical tertiary interpolationcoefficient k_(y)[4] based on the vertical coefficient based on thevertical displacement dy received from the vicinity selection section101.

For example, the horizontal tertiary interpolation coefficient k_(x)[4]can be calculated using the equation (1) below: $\begin{matrix}{{z = {{{dx} - i + 2}}}{{k_{x}\lbrack i\rbrack} = \left\{ \begin{matrix}{\left( {{3z^{3}} - {6z^{2}} + 4} \right)/6} & \left( {z < 1} \right) \\{\left( {{- z^{3}} + {6z^{2}} - {12z} + 8} \right)/6} & \left( {1<=z < 2} \right) \\0 & {Otherwise}\end{matrix} \right.}} & (1)\end{matrix}$

Also the vertical tertiary interpolation coefficient k_(y)[4] cantypically be calculated using the equation (2) below: $\begin{matrix}{{z = {{{dy} - j + 2}}}{{k_{y}\lbrack j\rbrack} = \left\{ \begin{matrix}{\left( {{3z^{3}} - {6z^{2}} + 4} \right)/6} & \left( {z < 1} \right) \\{\left( {{- z^{3}} + {6z^{2}} - {12z} + 8} \right)/6} & \left( {1<=z < 2} \right) \\0 & {Otherwise}\end{matrix} \right.}} & (2)\end{matrix}$

It is to be noted that any arbitrary calculation formula other than theequations (1), (2) shown in the above may be used for the calculation ofthe tertiary interpolation coefficients k_(x)[4] and k_(y)[4] so far asa sufficiently smooth interpolation can be obtained.

The products summation section 104 calculates an interpolation valueL_(cl)(p) of the position of interpolation p of the reduced imagelogL_(cl) by sum-of-products calculation using the neighboring pixelvalue a[4] [4], horizontal interpolation coefficient k_(x)[4] andvertical interpolation coefficient k_(y)[4], using the equation (3)below: $\begin{matrix}{{\log\quad{L_{cl}(p)}} = {\sum\limits_{i = 1}^{4}{\sum\limits_{j = 1}^{4}{{{a\lbrack i\rbrack}\lbrack j\rbrack} \cdot {k_{x}\lbrack i\rbrack} \cdot {k_{y}\lbrack j\rbrack}}}}} & (3)\end{matrix}$

Next, the gain value setting section 93 will be explained. The gainvalue setting section 93 is, as described in the above, used for settingthe gain value g(p) which is used for adjusting a degree of enhancementof regions other than the low-frequency region by the contrastenhancement section 94 in the succeeding stage. For a gain value ofg(p)=1, contrast is not enhanced nor suppressed by the contrastenhancement section 94. For a gain value of g(p)>1, contrast is enhancedcorresponding to the value. For a gain value of g(p)<1, contrast issuppressed corresponding to the value.

Setting of the gain value will be described. The contrast of the imagehas already been suppressed by the grayscale compression, wherein theamount of suppression depends on slope of the tone curve. For example,application of the tone curve having a small slope in view of effectinga strong grayscale compression means that contrast is stronglysuppressed. On the other hand, application of a straight line having aslope of 1 as the tone curve means that the image does not change, orcontrast is not suppressed.

The gain value setting section 93 therefore adopts an inverse 1/γ of therepresentative value γ of the tone curve, for the case where therepresentative value γ of the tone curve is smaller than 1, so that thegain value exceeds 1.

In another case where the input logarithmic luminance logL_(c)(p) isclose to the white level or black level, any contrast enhancementsimilar to that applied to the middle luminance region may undesirablyresult in loss of detail of the image due to clipping, therefore thegain value is adjusted so as to come closer to 1 as the inputlogarithmic luminance logL_(c)(p) comes closer to the white level orblack level.

That is, the gain value g(p) is calculated, assuming an inverse of therepresentative γ as 1/γ=g₀, using the equation (4) below:g(p)=1+(g ₀−1)×attn(p)  (4)where, attn(p) is an attenuating coefficient, and is calculated by theequation (5) below: $\begin{matrix}\begin{matrix}{{{attn}(p)} = {{attn}\left( {{\log\quad L_{gray}},{\log\quad L_{white}},{\log\quad{L_{c}(p)}}} \right)}} \\{= \left\{ \begin{matrix}{\frac{{\log\quad{L_{c}(p)}} - {\log\quad L_{gray}}}{{\log\quad L_{white}} - {\log\quad L_{gray}}}} & \begin{pmatrix}{{{2\log\quad L_{gray}} - {\log\quad L_{white}}} \leq} \\{{\log\quad{L_{c}(p)}} \leq {\log\quad L_{white}}}\end{pmatrix} \\1 & {Otherwise}\end{matrix} \right.}\end{matrix} & (5)\end{matrix}$

It is to be noted that, in the equation (5), logL_(gray) represents alogarithmic luminance expressing a moderate gray level, and logL_(white)represents a logarithmic luminance expressing the white clipping level(maximum white level), wherein both of which are preliminarily-setconstants.

FIG. 12 shows an exemplary configuration of the gain value settingsection 93. A divider 111 calculates the inverse 1/γ=g₀ of therepresentative value γ received from the preceding stage, and outputs itto a subtracter 112. The subtracter 112 calculates (g₀−1) and output itto a multiplier 118.

A subtracter 113 calculates difference (logL_(c)(p)−logL_(gray)) betweenthe logarithmic luminance logL_(c)(p) and the logarithmic luminancelogL_(gray) having a moderate gray level, and outputs it to a divider115. The subtracter 114 calculates difference (logL_(white)−logL_(gray))between the logarithmic luminance logL_(white) having a white clippinglevel and logarithmic luminance logL_(gray), and outputs it to a divider115. The divider 115 divides the output (logL_(c)(p)−logL_(gray)) fromthe subtracter 113 by the output (logL_(white)−logL_(gray)) from thesubtracter 114, and outputs the quotient to an absolute value calculator116. The absolute value calculator 116 calculates an absolute value ofthe output from the divider 115, and outputs it to a clipper 117. Theclipper 117 clips the output from the absolute value calculator 116 soas to adjust it to 1 when the output exceeds 1, but leaves it unchangedwhen the output does not exceed 1, and outputs the result as attn(p) toa multiplier 118.

The multiplier 118 multiplies the output from the subtracter 112 by theoutput from the clipper 117, and outputs the product to an adder 119.The adder 119 add 1 to the output from the multiplier 118, and outputsthe result as the gain value g(p) to the succeeding stage.

FIG. 13 in the next shows an exemplary configuration of the contrastenhancement section 94. A subtracter 121 calculates difference betweenthe logarithmic luminance logL_(c)(p) and the interpolation valuelogL_(cl) (p) of the reduced image, and outputs it to a multiplier 122.The multiplier 122 calculates a product of the output from thesubtracter 121 and gain value g(p), and outputs it to an adder 123. Theadder 123 adds the interpolation value logL_(cl) (p) of the reducedimage to the output of the multiplier 122, and outputs thuscontrast-corrected logarithmic luminance logL_(u)(p) to the succeedingstage.

It is to be noted now that the interpolation value logL_(cl)(p) of thereduced image is an interpolated value based on the m×n-pixel reducedimage, and therefore has only an extremely-low-frequency component ofthe image logL_(c) before being reduced.

That is, the output (logL_(c)(p)−logL_(cl)(p)) from the subtracter 121is equivalent to that obtained by subtracting only theextremely-low-frequency component from the original logarithmicluminance logL_(c)(p). The contrast-corrected logarithmic luminancelogL_(u)(p) is such as being obtained, as described in the above, bydividing the luminance signal into two categories ofextremely-low-frequency component and other components, and of these,the components other than the low-frequency components are enhanced bybeing multiplied by the gain value g(p), and by again synthesizing theboth using the adder 123.

As is known from the above, the contrast enhancement section 94 isdesigned so as to enhance the components ranging from thelow-to-middle-frequency region to high-frequency region, while excludingthe extremely-low-frequency region, using the same gain value g(p). Thecontrast-corrected logarithmic luminance logL_(u)(p) is therefore freefrom local overshoot at the edge portion which may otherwise be distinctwhen only the high-frequency region is enhanced, and is designed toobtain an image having a contrast enhanced very naturally to the eyes.

Next, the luminance range information calculation section 28 andluminance range normalization section 30 will be explained.

First, outline of the luminance range normalization processing will beexplained. A purpose of the grayscale compression by the DSP 7 residesin converting a wide-DR luminance image into a narrow-DR image adaptedto a dynamic range of a reproduction apparatus such as the display 11,and for this purpose, a tone curve adapted to the dynamic range of thereproduction apparatus is preliminarily prepared in the tone curvecorrection section 22. This makes it possible to subject most of thephotographed wide-DR luminance image appropriately to the grayscalecompression.

The dynamic range of incident light may, however, be not intrinsicallyso large depending on the subject to be photographed, and grayscalecompression processing of such image may result in an excessivegrayscale compression and consequently in confinement of the luminancein a range narrower than the dynamic range reproducible by thereproduction apparatus.

To avoid this, the luminance range normalization section 30 normalizes,as a processing in the final stage of the grayscale compressionprocessing, the luminance signal Y(p) after the gamma correction, sothat the dynamic range of the luminance signal Y(p) after the gammacorrection coincides with the dynamic range reproducible by thereproduction apparatus.

FIG. 14 shows a pattern of the luminance range normalization processingby the luminance range normalization section 30. In the line graph ofthis drawing, the abscissa plots gamma-corrected luminance Y before theluminance range normalization, the ordinate plots luminance Y_(n) afterthe luminance range normalization, and the grayscale conversion curve αexpresses a conversion table used for converting the luminance Y intoY_(n).

A method of determining the grayscale conversion curve α will bedescribed. The hatched graphic 131 shown under the line graph is anexemplary histogram of the luminance image Y before the luminance rangenormalization. In this example, in the stage after the gamma correctionbut before the luminance range normalization, a luminance image isobtained, of which grayscale being already compressed so as to have adynamic range narrower than a dynamic range ranging from a minimumluminance Y_(min) to a maximum luminance Y_(max) possibly generated bythe digital video camera 1.

Because output of the luminance image, while leaving the dynamic rangethereof unchanged, to the reproduction apparatus results in only aninefficient use of the dynamic range reproducible by the reproductionapparatus, the normalization is then carried out so that the luminancedistribution of the luminance image Y before the luminance rangenormalization extends over the entire portion of the dynamic range ofthe reproduction apparatus.

For this purpose, first a range [Y_(d), Y_(b)] over which the histogramgraphic 131 of the luminance image Y before the luminance rangenormalization distributes, is calculated as a luminance rangeinformation of the luminance image Y before the luminance rangenormalization. Then luminance values Y_(na) and Y_(ns) which fallslightly inward from the top and bottom ends of the luminance range[Y_(nb), Y_(nc)] of the reproduction apparatus are set, and thegrayscale conversion curve α is determined so as to correspondluminances {Y_(min), Y_(d), Y_(b), Y_(max)} on the abscissa withluminance values {Y_(nb), Y_(na), Y_(ns), Y_(nc)} on the ordinate.

Grayscale conversion using this grayscale conversion curve α issuccessful in obtaining the luminance image Y_(n) having a histogramform such as a hatched graphic 132 shown on the left hand side of theline graph.

A reason why the grayscale conversion curve α is determined so that theluminance range [Y_(d), Y_(b)] before the luminance range normalizationis mapped to the luminance range [Y_(na), Y_(ns)] slightly narrower thanthe luminance range [Y_(nb),Y_(nc)] of the reproduction apparatus isthat sharp luminance clipping at around the luminance values Y_(nb) andY_(nc) is prevented from appearing on the image.

It is to be noted herein that the luminance values Y_(na) and Y_(ns) arepreliminarily set with appropriate values based on the luminance valuesY_(nb) and Y_(nc).

Acquisition of the luminance range [Y_(d), Y_(b)] before the luminancerange normalization is carried out by the luminance range informationcalculation section 28, and calculation of the grayscale conversioncurve a and luminance Y_(n)(p) is executed by the luminance rangenormalization section 30.

FIG. 15 shows an exemplary configuration of the luminance rangeinformation calculation section 28. In the luminance range informationcalculation section 28, a decimation section 141 chooses the luminanceY(p) received from the gamma correction section 27 based on the pixelposition p. That is, only luminance values of images atpreliminarily-set pixel positions are supplied to a MIN sorting section142 and a MAX sorting section 145 in the succeeding stage.

The MIN sorting section 142 is configured so that k pairs of acombination of a comparison section 143 and a register 144 are arrangedin series, and so that the input luminance Y(p) values are held byregisters 144-1 to 144-k in an increasing order.

For example, the comparison section 143-1 compares the luminance Y(p)from the decimation section 141 and a value in the register 144-1, andupdates, when the luminance Y(p) from the decimation section 141 issmaller than the value in the register 144-1, the value in the register144-1 using the luminance Y(p) from the decimation section 141. On thecontrary, when the luminance Y(p) from the decimation section 141 is notsmaller than the value in the register 144-1, the luminance Y(p) fromthe decimation section 141 is supplied to the comparison section 143-2in the succeeding stage.

The comparison section 143-2 compares the luminance Y(p) from thecomparison section 143-1 and a value in the register 144-2, and updates,when the luminance Y(p) from the comparison section 143-1 is smallerthan the value in the register 144-2, the value in the register 144-2using the luminance Y(p) from the comparison section 143-1. On thecontrary, when the luminance Y(p) from the comparison section 143-1 isnot smaller than the value in the register 144-2, the luminance Y(p)from the comparison section 143-1 is supplied to the comparison section143-3 in the succeeding stage.

The same will apply also to the comparison sections 143-3 andthereafter, wherein after completion of input of the luminance Y(p) forone frame, the register 144-1 will have the minimum value Y_(min) of theluminance Y (p) held therein, and the registers 144-2 to 144-k will havethe luminance Y(p) values held therein in an increasing order, and theluminance Y(p) held in the register 144-k is output as the luminance Ydof the luminance range information to the succeeding stage.

The MAX sorting section 145 is configured so that k pairs of acombination of a comparison section 146 and a register 147 are arrangedin series, and so that the input luminance Y(p) values are held byregisters 147-1 to 147-k in a decreasing order.

For example, the comparison section 146-1 compares the luminance Y(p)from the decimation section 141 and a value in the register 147-1, andupdates, when the luminance Y(p) from the decimation section 141 islarger than the value in the register 144-1, the value in the register147-1 using the luminance Y(p) from the decimation section 141. On thecontrary, when the luminance Y(p) from the decimation section 141 is notlarger than the value in the register 147-1, the luminance Y(p) from thedecimation section 141 is supplied to the comparison section 146-2 inthe succeeding stage.

The comparison section 146-2 compares the luminance Y(p) from thecomparison section 146-1 and a value in the register 147-2, and updates,when the luminance Y(p) from the comparison section 146-1 is larger thanthe value in the register 147-2, the value in the register 147-2 usingthe luminance Y(p) from the comparison section 146-1. On the contrary,when the luminance Y(p) from the comparison section 146-1 is not largerthan the value in the register 147-2, the luminance Y(p) from thecomparison section 146-1 is supplied to the comparison section 146-3 inthe succeeding stage.

The same will apply also to the comparison sections 146-3 andthereafter, wherein after completion of input of the luminance Y (p) forone frame, the register 147-1 will have the maximum value Y_(max) of theluminance Y (p) held therein, and the registers 147-2 to 147-k will havethe luminance Y(p) values held therein in a decreasing order, and theluminance Y(p) held in the register 147-k is output as the luminanceY_(b) of the luminance range information to the succeeding stage.

Because the luminance Y(p) input to the MIN sorting section 142 and MAXsorting section 145 are decimated by the decimation section 141,appropriate adjustment of intervals of the decimating and the number ofsteps k of the MIN sorting section 142 and MAX sorting section 145 makesit possible to obtain luminance values Y_(d), Y_(b) which respectivelycorrespond with 1%, for example, of the upper and lower ends of theentire pixels in one frame.

FIG. 16 shows an exemplary configuration of the luminance rangenormalization section 30. The luminance range normalization section 30determines, as described in the above, the grayscale conversion curve α,and converts the gamma-corrected luminance Y(p) into the luminanceY_(n)(p) after the luminance range normalization using the grayscaleconversion curve α.

Because the grayscale conversion curve α is constructed from five linesegments as shown in FIG. 14, the luminance range normalization section30 discriminates to which segments the input luminance Y(p) belongs, andapplies one of five line segments composing the grayscale conversioncurve α to the input luminance Y(p) to there by convert it into theluminance Y_(n)(p) after the luminance range normalization.

A selector 151 of the luminance range normalization section 30 outputs,based on the luminance Y(p) input to an input terminal i, four luminancevalues out of the luminance values Y_(max), Y_(b), Y_(d), Y_(min),Y_(nc), Y_(ns), Y_(na) and Y_(nb) respectively input to input terminalsa to h from the output terminals j to m. Correlations thereamong areexpressed by the equation (6) below: $\begin{matrix}{\left\lbrack {j,k,l,m} \right\rbrack = \left\{ \begin{matrix}\left\lbrack {h,h,c,d} \right\rbrack & {i < d} \\\left\lbrack {h,g,c,d} \right\rbrack & {d \leq i < c} \\\left\lbrack {g,f,b,c} \right\rbrack & {c \leq i < b} \\\left\lbrack {f,e,a,b} \right\rbrack & {b \leq i < a} \\\left\lbrack {e,e,a,b} \right\rbrack & {a \leq i}\end{matrix} \right.} & (6)\end{matrix}$

A subtracter 152 calculates difference between an output from the outputterminal k and an output from the output terminal j, and outputs theresult to a divider 155. A subtracter 153 calculates difference betweenan output from the output terminal 1 and an output from a subtracter154, and outputs the result to the divider 155. The subtracter 154calculates difference between the luminance Y (p) and an output from theoutput terminal m, and outputs the result to a multiplier 156. Thedivider 155 calculates ratio of an output from the subtracter 152 and anoutput from the subtracter 153, and outputs the result to the multiplier156. The multiplier 156 calculates product of an output from the divider155 and an output from the subtracter 154, and outputs the result to anadder 157. The adder 157 adds an output from the output terminal j andan output from the multiplier 156, and outputs the result.

The output Y_(n)(p) from the adder 157 is expressed by the equation (7)below, which indicates the line segment of the grayscale conversioncurve α discriminated based on the gamma-corrected luminance Y(p).$\begin{matrix}{{Y_{n}(p)} = {{\frac{k - j}{l - m}\left( {{Y(p)} - m} \right)} + j}} & (7)\end{matrix}$

Here is an end of description on the individual portions composing theDSP 7 shown in FIG. 2.

Meanwhile, it is also possible to reduce the amount of calculation by asimpler circuit configuration, paying attention to that the averageluminance calculation section 63 of the tone curve correction section 22shown in FIG. 6 and the average luminance calculation section 72 of thereduced image generation section 23 execute the similar calculation.More specifically, the tone curve correction section 22, reduced imagegeneration section 23, reduced image memory 24 and contrast correctionsection 25, all of which shown in FIG. 2, can be combined to be providedas a composite section as shown in FIG. 17.

The composite section 160 can be a substitute for the tone curvecorrection section 22, reduced image generation section 23, reducedimage memory 24 and contrast correction section 25 shown in FIG. 2.

A LUT memory 161 of the composite section 160 has a LUT corresponded tothe tone curve as shown in FIG. 4 and a representative value γ₁ whichexpresses a slope of the tone curve preliminarily held therein. A tablereference section 162 corrects the logarithmic luminance logL(p)received from the preceding stage based on the LUT held by the LUTmemory 161 to thereby give a logarithmic luminance logL_(c′)(p), andoutputs it to a reduced image generation section 163 and a multiplier172.

The reduced image generation section 163 divides the logarithmicluminance image logL_(c), into m×n blocks, calculates an average valueof the logarithmic luminance logL_(c′)(p) of the pixels which belong tothe individual blocks to thereby generate an m×n-pixel first reducedimage, and makes a first reduced image memory 164 to store it.

The average luminance calculation section 63 calculates an average valueμ of pixel values of the previous frame's first reduced image held bythe first reduced image memory 164, and outputs it to a divider 166. Thedivider 166 divides a predetermined constant logL_(T) by the averagevalue μ to thereby calculate the representative value γ₂, and makes a γ₂memory 167 to store it. A multiplier 168 multiplies the individualpixels of the first reduced image held by the first reduced image memory164 by the representative value γ₂ held by the γ₂ memory 65 to therebygenerate a second reduced image logL_(cl), and makes a second reducedimage memory 169 to store it.

A multiplier 170 multiplies the logarithmic luminance logL_(c′)(p) ofthe current frame received from the table reference section 162 by theprevious frame's representative value γ₂ held by the γ₂ memory 167, tothereby calculates the logarithmic luminance logL_(c)(p) after the tonecurve correction. A multiplier 171 outputs a product of therepresentative values γ₁ and γ₂, as a representative value γ(=γ₁·γ₂), toa gain value setting section 172.

The gain value setting section 172 calculates a gain value g(p), whichdetermines an amount of contrast enhancement of the logarithmicluminance logL_(c)(p) of the current frame, based on the representativevalue γ for the preceding frame received from the multiplier 171, andthe logarithmic luminance logL_(c)(p) for the current frame receivedfrom the multiplier 170.

An interpolation position designation section 173 acquires a pixelposition p of the logarithmic luminance logL_(c)(p) of the current framereceived from the multiplier 170 in the preceding stage (also referredto as position of interpolation p, hereinafter), and outputs it to aninterpolation section 174. The interpolation section 174 calculates, byinterpolation, the pixel logL_(cl)(p) corresponded to the position ofinterpolation p, using the previous frame's second reduced imagelogL_(cl) held by the second reduced image memory 169, and outputs it toa contrast enhancement section 175.

The contrast enhancement section 175 calculates the logarithmicluminance logL_(u)(p) having an enhanced contrast in the componentsother than the low-frequency ones, with respect to the logarithmicluminance logL_(c)(p) of the current frame received from the multiplier170, based on the gain value g(p) and the interpolation valuelogL_(cl)(p) of the reduced image.

Use of the composite section 160 allows the average luminancecalculation section 165 to calculate an average value of the m×n-pixelfirst reduced image, and this is successful in reducing the volume ofcalculation as compared with the average luminance calculation section63 shown in FIG. 6 by which the average value is calculated for thepixels of the logarithmic luminance image of the original size. It istherefore possible to reduce the delay time ascribable to thecalculation.

Next, a general grayscale compression processing using the firstexemplary configuration of the DSP 7 applied with the composite section160 shown in FIG. 17 will be described referring to the flow chart ofFIG. 18.

In step S1, the DSP 7 converts the input wide-DR luminance image L ofthe current frame into the narrow-DR luminance image Y_(n), based on theintermediate information (second reduced image logL_(c)(p),representative value γ, luminance range information [Y_(d), Y_(b)])already calculated and held with respect to the previous frame's,wide-DR luminance image. The DSP 7 also calculates the intermediateinformation with respect to the wide-DR luminance image L of the currentframe.

In step S2, the DSP 7 updates the intermediate information with respectto the stored previous frame's wide-DR luminance image, using theintermediate information with respect to the calculated wide-DRluminance image L of the current frame.

In step S3, the DSP 7 discriminates whether any succeeding frame afterthe input wide-DR luminance image of the current frame is present ornot, and upon judgment of the presence, the process returns to step S1and processes thereafter are repeated. On the contrary, upon judgment ofthe absence of any succeeding frame, the grayscale compressionprocessing comes to the end.

Details of the processing on the pixel basis in step S1 will beexplained referring to the flow chart in FIG. 19. Processing of theindividual steps described below are executed with respect to a targetpixel (pixel position p) input according to order of raster.

In step S1, luminance L(p) of a target pixel (pixel position p) is inputto the DSP 7. In step S12, the logarithmic conversion section 21subjects the input luminance L(p) to the logarithmic conversion, andoutputs the obtained logarithmic luminance logL(p) to the compositesection 160. In step S13, the table reference section 162 of thecomposite section 160 corrects the logarithmic luminance logL(p)received from the logarithmic conversion section 21 to thereby obtainthe logarithmic luminance logL_(c′)(p) based on the LUT held by the LUTmemory 161, and outputs it to the reduced image generation section 163and multiplier 172. At the same time, the LUT memory 161 outputs therepresentative value γ₁ of the tone curve to the multiplier 171. Themultiplier 171 outputs a product of the representative values γ₁ and γ₂calculated based on the previous frame's first reduced image held by theγ₂ memory 167, as the representative value γ, to the gain value settingsection 172.

In step S14, the reduced image generation section 163 generates thefirst reduced image based on the logarithmic luminance logLc′(p) for oneframe after the tone curve correction. Based on the first reduced imagegenerated herein, the representative value γ₂ is calculated. Thegenerated first reduced image is also multiplied by the calculatedrepresentative value γ₂, and thereby the second reduced image logL_(cl)is generated.

In step S15, the multiplier 170 multiplies the logarithmic luminancelogL_(c′)(p) of the current frame received from the table referencesection 162 by the previous frame's representative value γ₂ held by theγ₂ memory 167, to thereby calculate the logarithmic luminancelogL_(c)(p) after the tone curve correction.

In step S16, the gain value setting section 172 calculates the gainvalue g(p), which determines the amount of contrast enhancement of thelogarithmic luminance logL_(c)(p) of the current frame, based on therepresentative value γ with respect to the preceding frame received fromthe multiplier 171 and the logarithmic luminance logL_(c)(p) of thecurrent frame received from the multiplier 170.

In step S17, the interpolation section 174 calculates, by interpolation,the pixel logL_(cl) (p) corresponded to the position of interpolation p,using the previous frame's second reduced image logL_(cl) held by thesecond reduced image memory 169, and outputs it to the contrastenhancement section 175. In step S18, the contrast enhancement section175 enhances the components other than low-frequency ones of thelogarithmic luminance logL_(c)(p) after the tone curve correction, basedon the interpolation value logL_(cl)(p) and gain value g(p) of thesecond reduced image, and outputs the obtained contrast-correctedlogarithmic luminance logL_(u)(p) to the logarithmic inversion section26 in the succeeding stage.

In step S19, the logarithmic inversion section 26 converts thecontrast-corrected logarithmic luminance logL_(u)(P) into the luminanceL_(u)(P) expressed by the normal axis, and outputs it to the gammacorrection section 27. In step S20, the gamma correction section carriesout a predetermined gamma correction, and outputs the obtained luminanceY(p) to the luminance range information calculation section 28 andluminance range normalization section 30.

In step S21, the luminance range information calculation section 28generates the luminance range information [Y_(d), Y_(b)] based on theluminance Y(p) for one frame. In step S22, the luminance rangenormalization section 30 normalizes the luminance Y(p) received from thegamma correction section 27, based on the previous frame's luminancerange information [Y_(d), Y_(b)] held by the luminance range informationmemory 29, to thereby calculate the luminance Y_(n)(p) In step S23, theluminance range normalization section 30 outputs the luminance Y_(n)(p)as a pixel value of the grayscale-compressed, narrow-DR luminance image.Here is an end of the detailed explanation of the processing in step S1shown in FIG. 18.

Next, details of the processing in step S2 in FIG. 18 will be explainedreferring to the flow chart in FIG. 20. In step S31, the reduced imagegeneration section 163 updates the first reduced image held by the firstreduced image memory 164, using the first reduced image generated basedon the logarithmic luminance logL_(c′)(p) for one frame after the tonecurve correction.

In step S32, the divider 166 divides a predetermined constant logL_(T)by the average value μ received from the average luminance calculationsection 165 to thereby calculate the representative value γ₂, andupdates the representative value γ₂ held by the γ₂ memory 167 using thecalculated representative value γ₂.

In step S33, the multiplier 168 multiplies the individual pixels of thefirst reduced image updated by the processing in step S31 and held bythe first reduced image memory 164, by the representative value γ₂updated by the processing in step S32 and held by the γ₂ memory 65, tothereby generate the second reduced image logL_(cl), and updates thesecond reduced image logL_(cl) held by the first reduced image memory169.

In step S34, the luminance range information calculation section 28updates the previous frame's luminance range information [Y_(d), Y_(b)]held by the luminance range information memory 29 using the luminancerange information [Y_(d), Y_(b)] calculated based on the luminance Y(p)for one frame. Here is the end of the detailed explanation of theprocessing of step S2 in FIG. 18.

Next, FIG. 21 shows an exemplary configuration of the DSP 7 adapted tothe wide-DR image, which is a color image. It is to be noted that thewide-DR image input to the DSP 7 according to the order of raster is notconfigured so that all of the pixels thereof individually have allcomponents of the R, G and B components, but is configured so as to haveeither one of the R, G and B components. The wide-DR image which is acolor image input to the second exemplary configuration of the DSP 7 isreferred to as a wide-DR color mosaic image, hereinafter. Which one ofthe R, G and B components do the individual pixels of the wide-DR colormosaic image have is determined by the position of the pixels.

It is defied now to denote a pixel value of the wide-DR color mosaicimage input to the DSP 7 in the order of raster as L(p).

In the second exemplary configuration of the DSP 7, a demosaicingsection 201 demosaics the pixel value L(p) for one frame in which eachpixel has a different color, so as to make all of the pixels have all ofR, G and B components, to thereby generate a color signal [R(p), G(p),B(p)], and outputs it to the color balance adjustment section 202. Animage constructed from the color signals output from the demosaicingsection 201 will be referred to as a wide-DR color image, hereinafter.

The color balance adjustment section 202 adjusts each of the R, G and Bcomponents so as to make the color balance of the entire imageappropriate, to thereby generate the color signal [R_(b)(P), G_(b)(P),B_(b)(P)]. It is to be noted that the demosaicing section 201 and colorbalance adjustment section 202 are such as those mounted on generaldigital video recorder equipped with a single-plate-type CCD imagesensor.

The logarithmic conversion section 203 subjects the color signal[R_(b)(P), G_(b)(P), B_(b)(P)] received from the color balanceadjustment section 202 to the logarithmic conversion, and outputs theobtained logarithmic color signal [logR_(b)(p), logG_(b)(p),logB_(b)(p)] to the tone curve correction section 204. The tone curvecorrection section 204 applies a preliminarily-obtained tone curve tothe input logarithmic color signal [logR_(b)(p), logG_(b)(p),logB_(b)(p)] and converts it in the direction of compressing thegrayscale, and outputs the obtained logarithmic color signal[logR_(b)(p), logG_(b)(p), logB_(b)(p)] to a reduced image generationsection 205 and a contrast correction section 207. The tone curvecorrection section 204 also outputs the representative value γ whichexpresses a slope of the applied tone curve to the contrast correctionsection 207.

The reduced image generation section 205 generates the reduced imagelogL_(cl) based on the logarithmic color signal [logR_(c)(p),logG_(c)(p), logB_(c)(p)] for one frame received from the tone curvecorrection section 204, and make the reduced image memory 206 to storeit.

The contrast correction section 207 corrects the contrast, weakened bythe tone curve correction, of the logarithmic color signal [logR_(c)(p),logG_(c)(p), logB_(c)(p)] of the current frame received from the tonecurve correction section 204, based on the representative value γ andthe previous frame s reduced image logL_(cl) held by the reduced imagememory 206, and outputs the obtained logarithmic color signal[logR_(u)(p), logG_(u)(p), logB_(u)(p)] to the logarithmic inversionsection 208. The logarithmic inversion section 208 subjects thecontrast-corrected logarithmic color signal [logR_(u)(p), logG_(u)(p),logB_(u)(p)] to the logarithmic inversion, and outputs the obtainedcolor signal [R_(u)(p), G_(u)(p), B_(u)(p)] expressed by the normal axisto the gamma correction section 209.

The gamma correction section 209 subjects the color signal [R_(u)(p),G_(u)(p), B_(u)(p)] received from the logarithmic inversion section 208to the gamma correction in consideration of the gamma characteristic ofa reproduction apparatus (e.g., display 11), and outputs the obtainedgamma-corrected color signal [R_(g)(p), G_(u)(p), B_(g)(p)] to theluminance information calculation section 210 and luminance rangenormalization section 212. The luminance information calculation section210 converts the [R_(g)(p), G_(u)(p), B_(g)(p)] for one frame, receivedfrom the gamma correction section 209, into the luminance Y(p),calculates the luminance range information for indicating thedistribution of the luminance Y (p), and makes the luminance rangeinformation memory 211 to store it. The luminance range informationdescribed herein refers to an information indicating a range ofdistribution of the luminance Y(p) for one frame, and is typicallycalculated as the luminance range information [Y_(d), Y_(b)] using theluminance Y_(d) closest to the darkness and the luminance Y_(b) closestto the brightness.

The luminance range normalization section 212 converts the color signal[R_(g)(p), G_(u)(p), B_(g)(p)] of the current frame received from thegamma correction section 209 so that the distribution range thereof cancoincide with a range expressible by the reproduction apparatus (e.g.,display 11), based on the previous frame's luminance range information[Y_(d), Y_(b)] held by the luminance range information memory 211, andoutputs the obtained color signal [R_(n)(p), G_(n)(p) B_(n)(p)] as thenarrow-DR image, which is a color image, to the succeeding stage.

As will be described below, the second exemplary configuration of theDSP 7 adapted to color image is almost similar to the first exemplaryconfiguration adapted to monochrome image shown in FIG. 2 except thatthe demosaicing section 201 and the color balance section 202 are added,but is slightly modified in the internal configurations of theindividual sections so as to be adapted to the color image.

FIG. 22 shows a first exemplary configuration of the tone curvecorrection section 204. In the first exemplary configuration, aluminance generation section 221 generates the logarithmic luminancelogL_(b)(p) by calculating a linear sum of the input logarithmic colorsignal [logR_(b)(p),logG_(b)(p),logB_(b)(p)], and outputs it to thesubtracters 222-R to 222-B, and to the table reference section 224.

The subtracter 222-R subtracts the logarithmic luminance logL_(b)(p)from the logarithmic color signal logR_(b)(p), and outputs the result toa multiplier 225-R. A LUT memory 223 has a LUT which corresponds to thetone curve as shown in FIG. 4 and a representative value γ indicating aslope of the tone curve previously held therein. The table referencesection 224 corrects the logarithmic luminance logL(p) into thelogarithmic luminance logL_(c)(p) using the LUT held by the LUT memory223, and output it to adders 226-R to 226-B.

The multiplier 225-R multiplies the output of the subtracter 222-R bythe representative value γ received from the LUT memory 223, and outputsit to the adder 226-R. The adder 226-R calculates a sum of the output ofthe multiplier 225-R and the logarithmic luminance logL_(c)(p), andoutputs the results as the logarithmic color signal logR_(c)(p) afterthe tone curve correction to the succeeding stage.

It is to be noted now that any constituents for processing the G and Bcomponents are similar to those for processing the above-described Rcomponent, so that explanations therefor will be omitted.

FIG. 23 shows a second exemplary configuration of the tone curvecorrection section 204. In the second exemplary configuration, aluminance generation section 231 calculates a linear sum of the inputlogarithmic color signal [logR_(b)(p), logG_(b)(p), logB_(b)(p)] tothereby generate the logarithmic luminance logL_(b)(p), and outputs itto an average luminance calculation section 232. The average luminancecalculation section 232 calculates an average value μ of the logarithmicluminance logL(p) for one frame, and outputs it to a divider 233. Thedivider 233 divides a predetermined constant by the average value μ tothereby calculates the representative value γ, and makes a γ memory 234to store it.

A multiplier 235-R multiplies the logarithmic color signal logR_(b)(p)of the current frame by the previous frame's representative value γ heldby the γ memory 234, to thereby calculates the logarithmic color signallogR_(c)(p) after the tone curve correction.

It is to be noted now that any constituents for processing the G and Bcomponents are similar to those for processing the above-described Rcomponent, so that explanations therefor will be omitted.

FIG. 24 shows a third exemplary configuration of the tone curvecorrection section 204. The third exemplary configuration is, so as tosay, a combination of the first exemplary configuration and secondexemplary configuration. In the third exemplary configuration, aluminance generation section 241 calculates a linear sum of thelogarithmic color signal [logR_(b)(p), logG_(b)(p), logB_(b)(p)] tothereby generate the logarithmic luminance logL_(b)(p), and outputs itto subtracters 242-R to 242-B, and to a table reference section 244.

The subtracter 242-R subtracts the logarithmic luminance logL_(b)(p)from the logarithmic color signal logR_(b)(p), and outputs the result toa multiplier 250-R. A LUT memory 243 has a LUT which corresponds to thetone curve as shown in FIG. 4 and the representative value γ indicatinga slope of the tone curve previously held therein. The table referencesection 244 corrects the logarithmic luminance logL(p) into thelogarithmic luminance logL_(c′)(p) using the LUT held by the LUT memory243, and outputs it to an average luminance calculation section 245 andto a multiplier 249.

The average luminance calculation section 245 calculates an averagevalue μ of the logarithmic luminance logL_(c′)(p) for one frame, andoutputs it to a divider 246. The divider 246 divides a predeterminedconstant logL_(T) by the average value μ to thereby calculate therepresentative value γ₂, and makes the γ₂ memory 247 to store it. Themultiplier 248 outputs a product of the representative values γ₁ and γ₂as the representative value γ (=γ₁·γ₂) to the contrast correctionsection 207 in the succeeding stage.

The multiplier 249 multiplies the logarithmic luminance logL_(c′)(p) ofthe current frame by the previous frame's representative value γ₂ heldby the γ₂ memory 247 to thereby calculate the logarithmic luminancelogL_(c)(p) after the tone curve correction, and outputs it to adders251-R to 251-B.

The multiplier 250-R multiplies the output of the subtracter 242-R bythe representative value γ received from the multiplier 248, and outputsthe result to the adder 251-R. The adder 251-R calculates a product ofan output of the multiplier 250-R and an output of the multiplier 249,and outputs the result as the logarithmic color signal logR_(c)(p) afterthe tone curve correction to the succeeding stage.

It is to be noted now that any constituents for processing the G and Bcomponents are similar to those for processing the above-described Rcomponent, so that explanations therefor will be omitted.

FIG. 25 in the next shows an exemplary configuration of the reducedimage generation section 205. A luminance generation section 261 of thereduced image generation section 205 calculates a linear sum of theinput logarithmic color signal [logR_(c)(p), logG_(c)(p), logB_(c)(p)]after the tone curve correction to thereby generate the logarithmicluminance logL_(c)(p), and outputs it to a sorting section 262.

A sorting section 262 sorts the logarithmic luminance logL_(c)(p) valuesaccording to blocks to which the luminance belongs when the entire imageis divided into m×n blocks, and then supplies them to the average valuecalculating sections 263-1 to 263-N (=m×n). For example, thoseclassified into the first block are supplied to the average valuecalculation section 263-1, and those classified into the second blockare supplied to the average value calculation section 263-2. The samewill apply also to the succeeding ones, and those classified into theN-th block are supplied to the average value calculation section 263-N.

The average value calculation means 263-i (i=1,2, . . . ,N) calculatesan average value of the logarithmic luminance logL_(c)(p) classifiedinto the i-th block, out of logarithmic luminance values logL_(c)(p) forone frame, and outputs it to a composition section 264. The compositionsection 264 generates an m×n-pixel reduced image logL_(cl) having, aspixel values, the average values of the logarithmic luminancelogL_(c)(p) respectively received from the average value calculationsection 263-i, and makes the reduced image memory 206 in the succeedingstage to store it.

FIG. 26 in the next shows an exemplary configuration of the contrastcorrection section 207. The luminance generation section 270 of thecontrast correction section 25 calculates a linear sum of the inputlogarithmic color signal [logR_(c)(p), logG_(c)(p), logB_(c)(p)] afterthe tone curve correction to thereby generate the logarithmic luminancelogL_(c)(p), and outputs it to an interpolation position designationsection 271 and a gain value setting section 273.

The interpolation position designation section 271 acquires a pixelposition p of the logarithmic luminance logL_(c)(p) (also referred to asposition of interpolation p, hereinafter), and outputs it to aninterpolation section 272. The interpolation section 272 calculates, byinterpolation, the pixel logL_(cl)(p) corresponded to the position ofinterpolation p, using the previous frame's second reduced imagelogL_(cl) held by the reduced image memory 206, and outputs it tosubtracters 274-R to 274-B, and to adders 276-R to 276-B.

The gain value setting section 273 calculates the gain value g(p), whichdetermines the amount of contrast enhancement of the logarithmicluminance logL_(c)(p) of the current frame, based on the representativevalue γ with respect to the preceding frame received from the tone curvecorrection section 22 and the logarithmic luminance logL_(c)(p) of thecurrent frame, and outputs it to multipliers 275-R to 275-B.

The subtracter 274-R subtracts the interpolation value logL_(cl)(p) fromthe logarithmic color signal logR_(c)(p) and outputs the result to themultiplier 275-R. The multiplier 275-R multiplies the output of thesubtracter 274-R by the gain value g(p), and outputs the result to theadder 276-R. The adder 276-R adds the interpolation value logL_(cl)(p)to the output of the multiplier 275-R, and outputs the obtainedlogarithmic color signal logR_(u)(p) after the contrast correction tothe succeeding stage.

It is to be noted now that any constituents for processing the G and Bcomponents are similar to those for processing the above-described Rcomponent, so that explanations therefor will be omitted.

FIG. 27 in the next shows an exemplary configuration of a compositesection 300 which can be a substitute for the tone curve correctionsection 204, reduced image generation section 205, reduced image memory206 and contrast correction section 207 shown in FIG. 21.

The luminance generation section 301 of the composite section 300calculates a linear sum of the input logarithmic color signal[logR_(b)(p), logG_(b)(p), logB_(b)(p)] to thereby generate thelogarithmic luminance logL_(b)(p), and outputs it to subtracter 302-R to302-B, and to a table reference section 304. The subtracter 302-Rsubtracts the logarithmic luminance logL_(b)(p) from the logarithmiccolor signal logR_(b)(p), and outputs the result to a multiplier 316-R.

A LUT memory 303 of the composite section 300 has a LUT corresponded tothe tone curve as shown in FIG. 4 and the representative value γ₁ whichexpresses a slope of the tone curve preliminarily held therein. A tablereference section 304 corrects the logarithmic luminance logL(p)received from the luminance generation section 301 based on the LUT heldby the LUT memory 303 to thereby give the logarithmic luminancelogL_(c′)(p), and outputs it to a multiplier 305 and a reduced imagegeneration section 306.

The multiplier 305 multiplies the logarithmic luminance logL_(c′)(p) ofthe current frame received from the table reference section 304 by theprevious frame's representative value γ₂ held by a γ₂ memory 167, tothereby calculate the logarithmic luminance logL_(c)(p) after the tonecurve correction, and outputs it to adders 317-R to 317-B.

The reduced image generation section 306 divides the logarithmicluminance image logL_(c′) into m×n blocks, calculates average values ofthe logarithmic luminance logL_(c′)(p) values of the pixels which belongto the individual blocks to thereby generate an m×n-pixel first reducedimage, and makes a first reduced image memory 307 to store it.

An average luminance calculation section 308 calculates an average valueμ of pixel values of the previous frame's first reduced image held bythe first reduced image memory 307, and outputs it to a divider 309. Thedivider 309 divides a predetermined constant logL_(T) by the averagevalue μ to thereby calculate the representative value γ₂, and makes a γ₂memory 310 to store it. A multiplier 311 calculates a product of therepresentative values γ₁ and γ₂ as the representative value γ(=γ₁·γ₂),and outputs it to a gain value setting section 315 and the multipliers316-R to 316-B.

The multiplier 312 multiplies the individual pixels of the first reducedimage held by the first reduced image memory 164 by the representativevalue γ₂ held by the γ₂ memory 310 to thereby generate the secondreduced image logL_(cl), and makes the second reduced image memory 313to store it.

An interpolation section 314 calculates, by interpolation, the pixellogL_(cl)(p) corresponded to the position of interpolation p (alsoreferred to as position of interpolation p, hereinafter) of thelogarithmic luminance logL_(c)(p) of the current frame received from themultiplier 170, using the previous frame's second reduced imagelogL_(cl) held by the reduced image memory 169, and outputs it tosubtracters 318-R to 318-B, and to adders 320-R to 320-B.

The gain value setting section 315 calculates the gain value g(p), whichdetermines the amount of contrast enhancement of the logarithmicluminance logL_(c)(p) of the current frame, based on the representativevalue γ with respect to the preceding frame received from the multiplier311 and the logarithmic luminance logL_(c)(p) of the current framereceived from the multiplier 305, and outputs it to multipliers 319-R to319-B.

The multiplier 316-R calculates a product of an output of the subtracter302-R and the representative value γ, and outputs it to the adder 317-R.The adder 317-R calculates a sum of the output of the multiplier 316-Rand the output of the multiplier 305, and outputs it to the subtracter318-R. The subtracter 318-R subtracts the interpolation valuelogL_(cl)(p) from the output of the adder 317-R, and outputs the resultto the multiplier 319-R. The multiplier 319-R multiplies the output ofthe subtracter 318-R by the gain value g(p), and outputs the result tothe adder 320-R. The adder 320-R calculates a sum of the output of themultiplier 319-R and the interpolation value logL_(cl)(p), and outputsthe obtained contrast-corrected logarithmic color signal logR_(u)(p) tothe succeeding stage.

It is to be noted now that any constituents for processing the G and Bcomponents are similar to those for processing the above-described Rcomponent, so that explanations therefor will be omitted.

Use of the composite section 300 allows the average luminancecalculation section 308 to calculate an average value of the m×n-pixelfirst reduced image, and this is successful in reducing the volume ofcalculation as compared with the average luminance calculation section245 shown in FIG. 24 by which the average value is calculated for thepixels of the logarithmic luminance image logL_(c) of the original size.It is therefore possible to reduce the delay time ascribable to thecalculation.

FIG. 28 in the next shows an exemplary configuration of the luminancerange information calculation section 210. In the luminance rangeinformation calculation section 210, a luminance generation section 331calculates a linear sum of the gamma-corrected color signal [R_(g)(p),G_(g)(p), B_(g)(p)] to thereby generate the luminance Y(p), and outputsit to a decimation section 332. The decimation section 332 chooses theluminance Y(p) received from the luminance generation section 331 basedon the pixel position p. That is, only luminance values of the pixels atpreliminarily-set pixel positions are supplied to a MIN sorting section333 and a MAX sorting section 336 in the succeeding stage.

The MIN sorting section 333 is configured so that k pairs of acombination of a comparison section 334 and a register 335 are arrangedin series, and so that the input luminance Y(p) values are held byregisters 335-1 to 335-k in an increasing order.

For example, the comparison section 334-1 compares the luminance Y(p)from the decimation section 332 and a value in the register 335-1, andupdates, when the luminance Y(p) from the decimation section 332 issmaller than the value in the register 335-1, the value in the register335-1 using the luminance Y(p) from the decimation section 332. On thecontrary, when the luminance Y(p) from the decimation section 332 is notsmaller than the value in the register 335-1, the luminance Y(p) fromthe decimation section 332 is supplied to the comparison section 334-2in the succeeding stage.

The same will apply also to the comparison sections 334-2 andthereafter, wherein after completion of input of the luminance Y(p) forone frame, the register 335-1 will have the maximum value Y_(min) of theluminance Y(p) held therein, and the registers 335-2 to 335-k will havethe luminance Y(p) values held therein in an increasing order, and theluminance Y(p) held in the register 335-k is output as the luminanceY_(d) of the luminance range information to the succeeding stage.

The MAX sorting section 336 is configured so that k pairs of acombination of a comparison section 337 and a register 338 are arrangedin series, and so that the input luminance Y(p) values are held byregisters 338-1 to 338-k in a decreasing order.

For example, the comparison section 337-1 compares the luminance Y(p)from the decimation section 332 and a value in the register 338-1, andupdates, when the luminance Y(p) from the decimation section 332 islarger than the value in the register 338-1, the value in the register338-1 using the luminance Y(p) from the decimation section 332. On thecontrary, when the luminance Y(p) from the decimation section 332 is notlarger than the value in the register 338-1, the luminance Y(p) from thedecimation section 332 is supplied to the comparison section 337-2 inthe succeeding stage.

The same will apply also to the comparison sections 337-2 andthereafter, wherein after completion of input of the luminance Y(p) forone frame, the register 338-1 will have the maximum value Y_(max) of theluminance Y(p) held therein, and the registers 338-2 to 338-k will havethe luminance Y(p) values held therein in a decreasing order, and theluminance Y(p) held in the register 338-k is output as the luminanceY_(b) of the luminance range information to the succeeding stage.

Because the luminance Y(p) values input to the MIN sorting section 333and MAX sorting section 336 are decimated by the decimation section 332,appropriate adjustment of intervals of the decimating and the number ofsteps k of the MIN sorting section 333 and MAX sorting section 336 makesit possible to obtain luminance Y_(d), Y_(b) values which correspondwith 1% or 0.1%, for example, of the upper and lower ends of the wholepixels in one frame.

Next, a general grayscale compression processing using the secondexemplary configuration of the DSP 7 applied with the composite section300 shown in FIG. 27 will be described referring to the flow chart ofFIG. 29.

In step S41, the DSP 7 (demosaicing section 201) demosaics a wide-DRcolor mosaic image to thereby generate a wide-DR color image, andoutputs the pixel value thereof, that is, a color signal [R(p), G(p),B(p)] to the color balance adjustment section 202 in the order ofraster. In step S42, the DSP 7 (color balance adjustment section 202)respectively adjust the R, G and B components so that a color balance ofthe entire image will become appropriate, to thereby generate the colorsignal [R_(b)(P), G_(b)(P), B_(b)(P)].

In step S43, the DSP 7 converts the color signal of the input wide-DRcolor image L of the current frame into the narrow-DR color image Y_(n),based on the intermediate information (second reduced image logL_(c)(p),representative value γ, and luminance range information [Y_(d), Y_(b)])already calculated and held with respect to the previous frame's,wide-DR color image. The DSP 7 also calculates the intermediateinformation with respect to the wide-DR color image L of the currentframe.

In step S44, the DSP 7 updates the intermediate information with respectto the stored previous frame's wide-DR color image, using theintermediate information with respect to the calculated wide-DR colorimage L of the current frame.

In step S45, the DSP 7 discriminates whether any succeeding frame afterthe input wide-DR color image of the current frame is present or not,and upon judgment of the presence, the process returns to step S41 andprocesses thereafter are repeated. On the contrary, upon judgment of theabsence of any succeeding frame, the grayscale compression processingcomes to the end.

Details of the processing on the pixel basis in step S42 will beexplained referring to the flow chart in FIG. 30. Processing of theindividual steps described below are executed with respect to a targetpixel (pixel position p) input according to order of raster.

In step S51, the color balance adjustment section 202 outputs thegenerated color signal [R_(b)(P), G_(b)(P), B_(b)(P)] to the logarithmicconversion section 203. In step S52, the logarithmic conversion section203 subjects the color signal [R_(b)(P), G_(b)(P), B_(b)(P)] to thelogarithmic conversion, and outputs the obtained logarithmic colorsignal [logR_(b)(p), logG_(b)(p), logB_(b)(p)] to the composite section300.

In step S53, the luminance generation section 301 of the compositesection 300 calculates a linear sum of the input logarithmic colorsignal [logR_(b)(p), logG_(b)(p), logB_(b)(p)] to thereby generate thelogarithmic luminance logL_(b)(p), and outputs it to the subtracters302-R to 302-B, and to the table reference section 304. In step S54,table reference section 304 corrects the input logarithmic luminancelogL(p) into the logarithmic luminance logL_(c′)(p) based on the LUTheld by the LUT memory 303, and outputs it to the multiplier 305 andreduced image generation section 306.

In step S55, the reduced image generation section 306 generates thefirst reduced image based on the logarithmic luminance logLc′ (p) forone frame after the tone curve correction. Herein the representativevalue γ₂ is calculated based on the generated first reduced image. Thesecond reduced image logL_(cl) is also generated herein by multiplyingthe generated first reduced image by thus calculated representativevalue γ₂.

In step S56, the multiplier 305 multiplies the logarithmic luminancelogL_(c′)(p) of the present image received from the table referencesection 304 by the previous frame's representative value γ₂ held in theγ₂ memory 310, to thereby calculate the logarithmic luminancelogL_(c)(p) after the tone curve correction.

In step S57, calculations are carried out for the R component by thesubtracter 302-R, multiplier 316-R and adder 317-R, to generate thelogarithmic color signal logR_(c)(p) after the tone curve correction.For G component, calculations are carried out by the subtracter 302-G,multiplier 316-G and adder 317-G, to generate the logarithmic colorsignal logG_(c)(p) after the tone curve correction. For B component,calculations are carried out by the subtracter 302-B, multiplier 316-Band adder 317-B, to generate the logarithmic color signal logB_(c)(p)after the tone curve correction.

In step S58, the gain value setting section 315 calculates the gainvalue g(p), which determines the amount of contrast enhancement of thelogarithmic luminance logL_(c)(p) of the current frame, based on therepresentative value γ with respect to the preceding frame received fromthe multiplier 311 and the logarithmic luminance logL_(c)(p) of thecurrent frame received from the multiplier 305. In step S59, theinterpolation section 314 calculates, by interpolation, the pixellogL_(cl) (p) corresponded to the position of interpolation p, using theprevious frame's second reduced image logL_(cl) held by the secondreduced image memory 313.

In step S60, calculations are carried out for the R component by thesubtracter 318-R, multiplier 319-R and adder 320-R, to generate thelogarithmic color signal logR_(u)(p) after the tone curve correction.For G component, calculations are carried out by the subtracter 318-G,multiplier 319-G and adder 320-G, to generate the logarithmic colorsignal logG_(u)(p) after the tone curve correction. For B component,calculations are carried out by the subtracter 318-B, multiplier 319-Band adder 320-B, to generate the logarithmic color signal logB_(u)(p)after the tone curve correction.

In step S61, the logarithmic inversion section 208 subjects thelogarithmic color signal [logR_(u)(p), logG_(u)(p), logB_(u)(p)] afterthe contrast correction to the logarithmic inversion to thereby generatethe color signal [R_(u)(p), G_(u)(p), B_(u)(p)] expressed by the normalaxis, and outputs it to the gamma correction section 209. In step S62,the gamma correction section 209 carries out a predetermined gammacorrection, and outputs the obtained gamma-corrected color signal[R_(g)(p), G_(g)(p), B_(g)(p)] to the luminance information calculationsection 210 and luminance range normalization section 212.

In step S63, the luminance generation section 331 of the luminance rangeinformation calculation section 210 generates the luminance Y(p) basedon the gamma-corrected color signal [R_(g)(p), G_(g)(p), B_(g)(p)]. Instep S64, the MIN sorting section 333 and MAX sorting section 336 of theluminance range information calculation section 210 calculate theluminance range information [Y_(d), Y_(b)] based on the luminance Y(p)for one frame.

In step S65, the luminance range normalization section 212 normalizesthe color signal [R_(g)(p), G_(g)(p), B_(g)(p)] received from the gammacorrection section 209 based on the previous frame's luminance rangeinformation [Y_(d), Y_(b)] held by the luminance range informationmemory 211, to thereby calculate the color signal [R_(n)(p), G_(n)(p),B_(n)(p)]. In step S66, luminance range normalization section 212outputs thus calculated color signal [R_(n)(p), G_(n)(p), B_(n)(p)] as apixel value of the grayscale-compressed, narrow-DR color image. Here isthe end of the detailed explanation of the processing of step S43 inFIG. 29.

Next, details of the processing in step S44 in FIG. 29 will be explainedreferring to the flow chart in FIG. 31. In step S71, the reduced imagegeneration section 306 updates the first reduced image held by the firstreduced image memory 307, using the first reduced image generated basedon the logarithmic luminance logL_(c′)(p) for one frame after the tonecurve correction.

In step S72, the divider 309 divides a predetermined constant logL_(T)by the average value μ received from the average luminance calculationsection 165 to thereby calculate the representative value γ₂, andupdates the representative value γ₂ held by the γ₂ memory 310 using thuscalculated representative value γ₂.

In step S73, the multiplier 312 multiplies the individual pixels of thefirst reduced image held by the first reduced image memory 307 updatedby the processing of step S71 by the representative value γ₂ held by theγ₂ memory 310 updated by the processing of step S72, to thereby generatethe second reduced image logL_(cl), and updates the second reduced imagelogL_(cl) held by the first reduced image memory 313.

In step S74, the luminance range information calculation section 210updates the previous frame's luminance range information [Y_(d), Y_(b)]held by the luminance range information memory 211, using the luminancerange information [Y_(d),Y_(b)] generated based on the [R_(g)(p),G_(g)(p), B_(g)(p)] for one frame. Here is the end of the detailedexplanation of the processing of step S44 in FIG. 29.

Here is the end of the detailed explanation of the second exemplaryconfiguration of the DSP 7.

It is to be noted, for example, that each of the average luminancecalculation section 51 shown in FIG. 5, average luminance calculationsection 63 shown in FIG. 6, average luminance calculation section 165shown in FIG. 17, average luminance calculation section 232 shown inFIG. 23 and average luminance calculation section 245 shown in FIG. 24was configured so that an average value of the luminance value wascalculated, wherein the calculation for finding the average value mayadopt weighted average. For example, a larger weight given to thecentral portion of an image rather than to the peripheral portionenables brightness correction while placing a stress on reflectance of asubject which resides in the central portion of the image.

The composite section 160 shown in FIG. 17 and the composite section 300shown in FIG. 27 have the memory for holding the generated first reducedimage and the memory for holding the second reduced image generated bymultiplying thus generated first reduced image by the representativevalue γ₂, wherein these two memories may be combined into one becausethe first reduced image is no more necessary to be held once the secondreduced image is generated.

If the present invention is applied to a digital video camera whichtakes a picture of a wide-DR image, compresses the grayscale and outputsit as an image displayable on a display having a narrow dynamic-range asin the present embodiment, it is made possible to realize the grayscalecompression processing by a configuration having only a largely reducedcapacity of memory (used for frame memory or delay line of the pixelseries data), which has otherwise been indispensable for theconventional grayscale compression technique, and it is also madepossible to obtain an output image which is by no means inferior to thatobtained by the grayscale compression processing conventionally beenrealized using a large-scale filtering.

This makes it possible to realize a high-quality and inexpensive digitalvideo camera which has never been realized.

The wide-DR image in the present embodiment was subjected to thegrayscale compression processing assuming the display 11 as areproduction apparatus, wherein it is also possible, for example, tocarry out the grayscale compression processing so as to be adapted todynamic ranges expressible by a monitor or a printed externallyconnected to the digital video camera 1.

FIG. 32 in the next shows an exemplary configuration of an imageprocessing system applied with the present invention. The imageprocessing system 501 is constructed from a video camera 502 for takinga picture of a subject and producing a wide-DR image L constructed frompixels having pixel values (luminance) having a dynamic range wider thanordinary ones, an image processing apparatus 510 for compressing thegrayscale of the wide-DR image L generated by the video camera 502 intoa grayscale range displayable by a display 511, and the display 11 fordisplaying a grayscale-compressed image L_(u) generated by the imageprocessing apparatus 510.

The video camera 502 is constructed from a lens 503 for condensing aphoto-image of a subject, a stop for adjusting an amount of light energyof the photo-image, a CCD image sensor for generating luminance signalsby photo-electric conversion of the condensed photo-image, apre-amplifier (Pre-amp.) 506 for removing noise component from thegenerated luminance signals, an AD converter (A/D) 507 for convertingthe luminance signals, removed with the noise component, typically intodigital data having a bit width of 14 to 16 bits or around, and an I/Ointerface (I/O) 508 for outputting the wide-DR image, constructed frompixels having digitized luminance, to the image processing apparatus510.

FIG. 32 shows overall operations of the image processing system 1. Instep S101, the video camera 502 takes a picture of the subject,generates a correspondent wide-DR image L, and outputs it to the imageprocessing apparatus 510. In step S102, the image processing apparatus510 subjects the wide-DR image L to the grayscale compression processingto thereby generate the grayscale-compressed image L_(u), and outputs itto the display 511. In step S103, the display 511 displays thegrayscale-compressed image L_(u).

FIG. 34 in the next shows a first exemplary configuration of the imageprocessing apparatus 510. Atone curve correction section 521 of theimage processing apparatus 510 corrects the wide-DR image L receivedform the video camera 502 in the direction of compressing the grayscalebased on a preliminarily-obtained tone curve, and outputs the resultanttone-curve-corrected image L_(c) to a smoothed luminance generationsection 522, a gain value setting section 523, and a contrast correctionsection 524. It is to be noted that the tone-curve-corrected image L_(c)has a compressed grayscale, and a weakened contrast ascribable to thecompressed grayscale. The tone curve correction section 521 also outputsthe representative value γ, which expresses a slope of the tone curveused for the correction, to the gain value setting section 523.

FIG. 35 shows an exemplary configuration of the tone curve correctionsection 521. A LUT memory 531 of the tone curve correction section 521preliminarily holds a lookup table (referred to as LUT, hereinafter)which corresponds with a monotonously-increasing tone curve as shown inFIG. 36, and a representative value γ which expresses a slope of thetone curve. It is also allowable that a function corresponded to thetone curve is held in place of the LUT. A table reference section 532corrects the wide-DR image L based on the LUT held by the LUT memory 531to thereby obtain the tone-curve-corrected image L_(c).

FIG. 36 shows an example of the tone curve, wherein the abscissa plotsluminance of the wide-DR image L, and the ordinate plots luminance ofthe tone-curve-corrected image L_(c) after the correction, respectivelyon the logarithmic axes normalized over a range of [0, 1]. The tonecurve shown in FIG. 36 does not correct the luminance value of thenormalized wide-DR image when the value exceeds 0.5, but corrects theluminance value of the normalized wide-DR image value when the value issmaller than 0.5 so that a smaller value is corrected by a larger amountof correction. That is, the correction is carried out so as to avoidmaximum shadowing of the dark area in the image when appeared on thedisplay 511. The representative value γ which expresses a slope of thetone curve can be defined by an average value of slopes respectivelyfound over the entire regions of the luminance. For example, therepresentative value of the tone curve shown in FIG. 36 is γ=0.94.

Referring now back to FIG. 34, the smoothed luminance generation section522 smoothes the luminance of the tone-curve-corrected image L_(c), andoutputs the luminance L_(cl)(p) of the obtained smoothed image to thecontrast correction section 24. FIG. 37 shows an exemplary configurationof the smoothed luminance generation section 22.

A reduced image generation section 541 of the smoothed luminancegeneration section 522 classifies pixels of the tone-curve-correctedimage L_(c) received from the tone curve correction section 521 into m×nblocks depending on positions of the pixels, and generates the reducedimage L_(cl) having an average value of the luminance of the pixelsclassified into the individual blocks as its own pixels. A reduced imagememory 542 holds thus generated m×n-pixel reduced image L_(cl). Aninterpolation section 543 calculates, by interpolation, the luminance ofthe pixel position sequentially specified using the pixels of thereduced image held by the reduced image memory 542, and outputs theobtained interpolation value L_(cl)(p) as luminance of the pixels of thesmoothed image to the contrast correction section 524. It is to be notedherein that p=(x,y) is a coordinate or vector expressing the pixelposition. Size of the smoothed image output from the interpolationsection 543 is equivalent to that of the tone-curve-corrected imageL_(c).

That is, in the smoothed luminance generation section 522, thetone-curve-corrected image L_(c) is reduced to generate the reducedimage L_(cl), and by using the held reduced image L_(cl), the luminanceof the smoothed image is calculated by interpolation operation in apixel-wise manner.

Although it was conventionally necessary to adopt a relativelylarge-scale filtering for an effective grayscale compression processing,the smoothed luminance generation section 522 needs only the reducedimage memory 542 for holding the m×n-pixel reduced image.

FIG. 38 shows an exemplary configuration of the reduced image generationsection 541 shown in FIG. 37. A sorting section 551 of the reduced imagegeneration section 541 sorts the pixels of the tone-curve-correctedimage L_(c) received from the tone-curve correction section 521 into m×nblocks depending on the pixel position, and then supplies them toaverage value calculating sections 552-1 to 552-N (=m×n). For example,those classified into the first block are supplied to average valuecalculation section 552-1, and those classified into the second blockare supplied to the average the value calculation section 552-2. Thefollowing description adopts a simple notation of average valuecalculation section 552 when there is no need of discrimination of theindividual average value calculation sections 552-1 to 552-N.

The average value calculation section 552-i (i=1,2, . . . ,N) calculatesan average value of the luminance of the pixels of thetone-curve-corrected image L_(c) classified into the i-th block, andoutputs it to a composition section 553. The composition section 553generates an m×n-pixel reduced image logL_(cl) having, as a pixel value,the average value of the luminance respectively received from theaverage value calculation means 552-i.

FIG. 39 shows an exemplary configuration of the average valuecalculation section 552 shown in FIG. 38. An adder 561 of the averagevalue calculation section 552 adds the luminance of thetone-curve-corrected image L_(c) received from the sorting section 551in the preceding stage to a value held by a register (r) 562, to therebyupdate the value held by the register (r) 562. A divider 563 divides avalue finally held by the register 562 by the number of pixels Qcomposing one block, to thereby calculate an average value of theluminance of Q pixels classified into one block.

FIG. 40 shows an exemplary configuration of the interpolation section543 shown in FIG. 37. A vicinity selection section 571 of theinterpolation section 543 acquires, upon reception of the position ofinterpolation p, 4×4-pixel luminance a[4] [4] in the vicinity of theposition of interpolation p, based on the m×n-pixel reduced imagelogL_(cl) held by the reduced image memory 542, and outputs it to theproducts summation section 574.

A notation of a[i] [j] herein means that pixel value a is an i×jtwo-dimensional arrangement data. The vicinity selection section 571outputs horizontal displacement dx and vertical displacement dy betweenthe acquired luminance a [4] [4] and position of interpolation p to ahorizontal coefficient calculation section 572 or a vertical coefficientcalculation section 573, respectively.

It is to be noted that relations among the position of interpolation p,neighboring luminance a[4] [4] and amounts of displacement dx, dy aresimilar to those described in the above referring to FIG. 11, so thatthe explanation therefor will be omitted.

The horizontal coefficient calculation section 572 calculates a tertiaryinterpolation coefficient k_(x)[4] in the horizontal direction based onthe horizontal displacement dx received from the vicinity selectionsection 71. Similarly, the vertical coefficient calculation section 573calculates a tertiary interpolation coefficient k_(y)[4] in the verticaldirection based on the vertical displacement dy received from thevicinity selection section 571.

The tertiary interpolation coefficient k×[4] in the horizontal directionis typically calculated by using the equation (1) described in theabove.

The tertiary interpolation coefficient k_(y)[4] in the verticaldirection is typically calculated by using the equation (2) described inthe above.

It is to be noted that any arbitrary calculation formula other than theequations (1), (2) shown in the above may be used for the calculation ofthe tertiary interpolation coefficients k_(x)[4] and k_(y)[4] so far asa sufficiently smooth interpolation can be obtained.

The products summation section 574 calculates an interpolation valueL_(cl)(p) of the position of interpolation p of the reduced image L_(cl)by sum-of-products calculation using the neighboring pixel value a[4][4], horizontal interpolation coefficient k_(x)[4] and verticalinterpolation coefficient k_(y)[4], using the equation (3) described inthe above.

Referring now back to FIG. 34, the gain setting section 523 calculatesthe gain value g(p), which is used for adjusting the amount ofcorrection of the contrast of the luminance logL_(c)(p) of the smoothedimage in the contrast correction section 524, for the individual pixelpositions based on the representative value γ received from the tonecurve correction section 521, and outputs it to the contrast correctionsection 524.

The gain value g(p) will be explained below. For a gain value of g(p)=1,contrast is not enhanced nor suppressed by the contrast enhancementsection 524. For a gain value of g(p)>1, contrast is enhancedcorresponding to the value. On the contrary, for a gain value of g(p)<1,contrast is suppressed corresponding to the value.

Outline of the gain value setting by the gain setting section 523 issimilar to the gain value setting by the above-described gain settingsection 93, so that the explanation therefor will be omitted.

FIG. 41 shows an exemplary configuration of the gain value settingsection 523. A divider 581 calculates the inverse 1/γ=g₀ of therepresentative value γ received from the preceding stage, and outputs itto a subtracter 582. The subtracter 582 calculates (g₀−1) and output itto a multiplier 588.

A subtracter 583 calculates difference (L_(c)−L_(gray)) between theindividual luminance of the tone-curve-corrected image L_(c) and theluminance L_(gray) having a moderate gray level, and outputs it to adivider 585. The subtracter 584 calculates difference(L_(white)−L_(gray)) between the luminance L_(white) having a whiteclipping level and luminance L_(gray), and outputs it to a divider 585.The divider 585 divides the output (L_(c)−L_(gray)) of the subtracter583 by the output (L_(white)−L_(gray)) of the subtracter 584, andoutputs it to an absolute value calculator 586. The absolute valuecalculator 586 calculates an absolute value of the output from thesubtracter 585, and outputs it to a clipper 587. The clipper 587 clipsthe output from the absolute value calculator 586 so as to adjust it to1 when the output exceeds 1, but leaves it unchanged when the outputdoes not exceed 1, and outputs the result as attn(p) to a multiplier588.

The multiplier 588 multiplies the output from the subtracter 582 by theoutput from the clipper 587, and outputs the product to an adder 589.The adder 589 adds 1 to the output from the multiplier 588, and outputsthe result as the gain value g(p) to the succeeding stage.

Referring now back to FIG. 34, the contrast correction section 524enhances the contrast of the tone-curve-corrected image L_(c) which haspreviously been weakened, based on the gain value g(p) for theindividual pixel positions p received from the gain value settingsection 523 and the luminance L_(cl)(p) of the smoothed image receivedfrom the smoothed luminance generation section 522, to thereby generatethe grayscale-compressed image L_(u).

FIG. 42 shows an exemplary configuration of the contrast correctionsection 524. A subtracter 591 of the contrast correction section 524calculates difference (L_(c)(p)−L_(cl)(p)) between the luminance L_(c)(p) of the individual pixels of the tone-curve-corrected image L_(c) andthe luminance of the corresponded pixels of the smoothed image (i.e,interpolation value L_(cl)(p) of the reduced image), and outputs it to amultiplier 592. The multiplier 592 calculates a product of an output ofthe subtracter 591 and the gain value g(p) received from the gain valuesetting section 523, and outputs the result to an adder 593. The adder593 adds the luminance of the smoothed image (interpolation valueL_(cl)(p) of the reduced image) to an output of the multiplier 592, andoutputs the obtained luminance L_(u)(p), as the luminance of pixelscomposing the contrast-corrected, grayscale-compressed image L_(u), tothe succeeding stage.

It is to be noted now that the luminance of pixels of the smoothed image(interpolation value L_(cl)(p) of the reduced image) is an interpolatedvalue based on the pixels of the m×n-pixel reduced image L_(cl), andtherefore has only an extremely-low-frequency component of the imageL_(c) before being reduced.

Hence, the output (L_(c)(p)−L_(cl)(p)) of the subtracter 591 isequivalent to that obtained by subtracting only theextremely-low-frequency component from the original tone-curve-correctedimage logL_(c). The luminance L_(u)(p) of the contrast-corrected,grayscale-compressed image is such as being obtained, as described inthe above, by dividing the luminance signal into theextremely-low-frequency component and other components, and of these,the components other than the low-frequency components (output of thesubtracter 591) are enhanced in the contrast by being multiplied by thegain value g(p), and by again synthesizing the both using the adder 593.

As described in the above, the contrast correction section 524 isconfigured so as to enhance the components in thelow-to-middle-frequency region and high-frequency region, except for theextremely-low-frequency region, using the same gain value g(p).Therefore, it is made possible to obtain an image having the contrastenhanced very naturally to the eyes, without generating a localovershoot of the edge portion which may otherwise be distinct when onlythe high-frequency component is enhanced.

Next, the grayscale-compressed image generation processing by the imageprocessing apparatus 510 according to the first exemplary configuration(i.e., the processing in step S102 described in the above referring tothe flow chart in FIG. 33) will be explained referring to the flow chartin FIG. 43.

In step S111, the tone curve correction section 521 corrects theluminance of the wide-DR image L received from the video camera 502based on the preliminarily-obtained LUT, and outputs the obtainedtone-curve-corrected image L_(c) to the smoothed luminance generationsection 522, gain value setting section 523, and contrast correctionsection 524. The tone curve correction section 521 also outputs therepresentative value γ, which expresses a slope of the tone curve usedfor the correction, to the gain value setting section 523.

In step S112, the smoothed luminance generation section 522 shrinks thetone-curve-corrected image L_(c) to thereby generate the reduced imageL_(cl), and further calculates the luminance L_(cl)(p) of pixels of thesmoothed image based on the interpolation operation using the pixels ofthe reduced image L_(cl), and outputs the result to the contrastcorrection section 524.

In step S113, the gain setting section 523 calculates the gain valueg(p) which is used for adjusting the amount of correction of thecontrast of the luminance L_(c)(p) of the smoothed image in the contrastcorrection section 524, for the individual pixel positions based on therepresentative value γ received from the tone curve correction section521, and outputs it to the contrast correction section 524.

It is to be noted that the processing in step S112 and step S113 can becarried out in parallel.

In step S114, the contrast correction section 524 corrects the luminanceof the tone-curve-corrected image L_(c), based on the gain value g(p)for the individual pixel positions p received from the gain valuesetting section 523, and the luminance L_(cl)(p) of the smoothed imagereceived from the smoothed luminance generation section 522, to therebycalculate the luminance L_(u)(p) of pixels of the grayscale-compressedimage L_(u). The contrast-corrected grayscale-compressed image L_(u)that is obtained as described above is therefore obtained as an imagehaving the contrast enhanced very naturally to the eyes, withoutgenerating a local overshoot of the edge portion which may otherwise bedistinct when only the high-frequency component is enhanced. Here is theend of the explanation on the grayscale-compressed image generationprocessing by the first exemplary configuration of the image processingapparatus 510.

FIG. 44 in the next shows a second exemplary configuration of the imageprocessing apparatus 510. The second exemplary configuration is such asbeing configured so that a logarithmic conversion section 601 foreffecting the logarithmic conversion of the luminance of the wide-DRimage L received from the video camera 501 is provided in the precedingstage of the tone curve correction section 521 in the first exemplaryconfiguration shown in FIG. 34, and so that a logarithmic inversionsection 602 for effecting the logarithmic inversion of the output of thecontrast correction section 524 is provided in the succeeding stage ofthe contrast correction section 524 in the first exemplaryconfiguration.

Any constituents other than the logarithmic conversion section 601 andlogarithmic inversion section 602 composing the second exemplaryconfiguration of the image processing apparatus 510 are equivalent tothose in the first exemplary configuration shown in FIG. 34, and aregiven with the same reference numerals, so that the explanationstherefor will be omitted. It is to be noted herein in the secondexemplary configuration that the sections from the tone curve correctionsection 521 to the contrast correction section 524 individuallyprocesses the luminance after logarithmic conversion.

The tone curve correction section 521 in the second exemplaryconfiguration adopts the tone curve such as shown in FIG. 4, forexample. Application of the monotonously-increasing moderateinverse-S-shaped curve as shown in FIG. 4 will not give so strong effectof grayscale compression in the high luminance region and low luminanceregion, so that it is possible to obtain a desirable tone with lessdegree of whiteout or blackout even after the grayscale compression. Onthe contrary, the grayscale compression will strongly affect the middleluminance region, but this means that the contrast correction can fullybe applied with the middle luminance region, and results in a desirablegrayscale-compressed image L_(u) with a less degree of contrastcorrection also in the middle luminance range. The tone curve herein hasa representative value γ of 0.67.

Next, details of the grayscale-compressed image generation processingaccording to the second exemplary configuration of the image processingapparatus 510 will be explained referring to the flow chart in FIG. 45.

In step S121, the logarithmic conversion section 601 subjects thewide-DR image L received from the video camera 502 2 into logarithmicconversion, and outputs the obtained logarithmic wide-DR image logL tothe tone curve correction section 521.

In step S122, the tone curve correction section 521 corrects theluminance of the logarithmic wide-DR image logL, typically based on thepreliminarily-obtained LUT, which corresponds to the tone curve shown inFIG. 4, and outputs the obtained logarithmic tone-curve-corrected imagelogL_(c) to the smoothed luminance generation section 522, gain valuesetting section 523, and contrast correction section 524. The tone curvecorrection section 521 also outputs the representative value γ, whichexpresses a slope of the tone curve used for the correction, to the gainvalue setting section 523.

In step S123, the smoothed luminance generation section 522 shrinks thelogarithmic tone-curve-corrected image logL_(c) to thereby generate thelogarithmic reduced image logL_(cl), and further calculates theluminance logL_(cl) (p) of pixels of the logarithmic smoothed image bythe interpolation operation using pixels of the logarithmic reducedimage logL_(cl), and outputs the result to the contrast correctionsection 524.

In step S124, the gain setting section 523 calculates the gain valueg(p) which is used for adjusting the amount of correction of thecontrast of the luminance logL_(c)(p) of the logarithmic smoothed imagein the contrast correction section 524, for the individual pixelpositions based on the representative value γ received from the tonecurve correction section 521, and outputs it to the contrast correctionsection 524.

It is to be noted that the processing in step S123 and step S124 can becarried out in parallel.

In step S125, the contrast correction section 524 corrects the luminanceof the logarithmic tone-curve-corrected image logL_(c) based on the gainvalue g(p) for the individual pixel positions p received from the gainvalue setting section 523, and the luminance logL_(cl) (p) of thelogarithmic smoothed image received from the smoothed luminancegeneration section 522, to thereby calculate the luminance logL_(u)(p)of pixels of the logarithmic grayscale-compressed image logL_(u), andoutputs it to the logarithmic inversion section 602.

In step S126, the logarithmic inversion section 602 subjects theluminance logL_(u)(p) of pixels of the logarithmic grayscale-compressedimage logL_(u) to the logarithmic inversion, and outputs the obtainedL_(u)(p) as the luminance of pixels of the grayscale-compressed imageL_(u).

Because the contrast-corrected grayscale-compressed image L_(u) that isobtained as described above will not give so strong effect of grayscalecompression in the high luminance region and low luminance region, sothat it is possible to obtain a desirable tone with less degree ofwhiteout or blackout even after the grayscale compression. On thecontrary, the grayscale compression will strongly affect the middleluminance region, but this means that the contrast correction can fullybe applied with the middle luminance region, and results in a desirablegrayscale-compressed image L_(u) with a less degree of contrastcorrection also in the middle luminance range. Here is the end of theexplanation of the grayscale-compressed image generation processingaccording to the second exemplary configuration of the image processingapparatus 510.

As has been described in the above, the image processing apparatus 510according to one embodiment of the present invention makes it possibleto convert a wide-DR image, having a dynamic range of luminance widerthan the usual, to a grayscale-compressed image displayable on thedisplay 111 having only a narrow dynamic-range of displayable luminance,based on a configuration largely reduced in a large capacity of memory(used as a frame memory and data delay line) which has beenindispensable for the conventional grayscale compression technique,without ruining the nice-looking property. It is also made possible toobtain a grayscale-compressed image which is by no means inferior tothat obtained by the grayscale compression processing conventionallybeen realized using a large-scale filtering.

Of course, the image processing apparatus 510 can convert the wide-DRimage into the grayscale-compressed image, while being adapted to thedynamic range expressible on printers and projectors, besides thedisplay 511.

The present invention is typically applicable to image signal processingcircuit incorporated not only in photographing apparatuses such asdigital video camera and digital still camera, but also in expressionapparatuses such as display, printer, projector and so forth.

A series of processing described in the above can be executed on thehardware basis, but can be executed also on the software basis. For thecase where the series of processing is executed on the software basis, aprogram composing the software is installed from a recording medium on acomputer incorporated in a dedicated hardware, or, for example, to ageneral-purpose personal computer which can execute a variety offunctions after being installed with a variety of programs.

FIG. 46 shows an exemplary configuration of a general-purpose personalcomputer. The personal computer 620 has a CPU (Central Processing Unit)621 incorporated therein. The CPU 621 is connected with an input/outputinterface 625 via a bus 624. The bus 624 is connected with a ROM (ReadOnly Memory) 622 and a RAM (Random Access Memory) 623.

The input/output interface 625 is connected with an input section 626which is constructed from an input apparatus such as keyboard, mouse andso forth through which the user enters operational commands, an outputsection 627 for outputting processing operation screen or resultantimage of processing on a display apparatus, a storage section 628structured with a hard disk drive and so forth for storing programs ofvarious data, and an I/O interface 629 used for image data communicationwith the video camera 502 or the like. It is also connected with a drive630 used for writing/reading of data to or from a recording medium suchas a magnetic disk 631 (including flexible disk), optical disk 632(including CD-ROM (Compact Disc-Read Only Memory) and DVD (DigitalVersatile Disc)), a magneto-optical disk 633 (including MD (Mini Disc)),or a semiconductor memory 634.

The CPU 621 executes various processing according to a program stored inthe ROM 622, or a program read out from any of the media ranging fromthe magnetic disk 631 to semiconductor memory 634 and installed on thestorage section 628, and loaded from the storage section 628 to the RAM623 from the storage section 628. The RAM 623 has, properly storedtherein, also data necessary for the CPU 621 to carry out variousprocessing.

It is to be understood that, in this patent specification, the steps fordescribing the program recorded in a recording medium includes not onlyprocessing carried out according to the given order on the time-seriesbasis, but may also contain processing carried out in parallel orindividually in place of those always processed on the time-seriesbasis.

It is to be understood that the whole contents of the claims,specifications, drawings and abstracts of Japanese Patent ApplicationNo. 2002-003134 (filed on Jan. 9, 2003) and No. 2003-003135 (filed onJan. 9, 2003) are referred to and incorporated herein in thisspecification.

INDUSTRIAL APPLICABILITY

As has been described in the above, the present invention makes itpossible to realize a grayscale compression technique which requiresonly a small memory capacity to be consumed and a light load ofcalculation, allows easy hardware construction, and ensures a largegrayscale compression effect.

It is also made possible to appropriately enhance the contrast of imageusing a smaller capacity of memory, based on a less amount ofcalculation, and based on a simple hardware construction.

1. An image processing apparatus characterized by comprising: reduced image generation means for generating a reduced image from an input image; correction information acquisition means for acquiring correction information of the input image based on the reduced image; and grayscale conversion means for converting grayscale of the input image; wherein the grayscale conversion means corrects contrast of the input image using the correction information, as a processing to be performed before and/or after the grayscale is converted.
 2. The image processing apparatus according to claim 1, characterized by further comprising: smoothing means for generating a smoothed image having luminance L_(c) of pixels composing the input image smoothed based on interpolation calculation using pixels composing the reduced image, wherein the grayscale conversion means generate a contrast-corrected image based on luminance L_(c) of pixels composing the image, luminance L₁ of pixels composing the smoothed image, and a predetermined gain value g.
 3. The image processing apparatus according to claim 1, characterized by further comprising: smoothing means for generating a smoothed image having luminance L_(c) of pixels composing the input image smoothed based on interpolation calculation using pixels composing the reduced image; and gain value setting means for setting a gain value g used for correcting the contrast; wherein the grayscale conversion means generate a contrast-corrected image based on luminance L_(c) of pixels composing the input image, luminance L₁ of pixels composing the smoothed image, and a predetermined gain value g; and the gain value setting means can be configured so as to set the gain value g based on input initial gain value g₀, reference gain value 1, and an attenuation value attn(Th₁, Th₂, L_(c)) calculated using a first luminance threshold value Th₁, a second luminance threshold value Th₂, and luminance L_(c) of pixels composing the input image.
 4. The image processing apparatus according to claim 1, characterized by further comprising: conversion means for generating a tone-converted image by converting luminance L of pixels composing the input image based on a conversion function; smoothing means for generating a smoothed image by smoothing luminance L_(c) of pixels composing the tone-converted image; and gain value setting means for setting a gain value g used for correcting the contrast based on an initial gain value g₀ which expresses an inverse 1/γ of a slope γ of the conversion function; wherein the contrast correction means generate a contrast-corrected image based on luminance L_(c) of pixels composing the tone-converted image, luminance L₁ of pixels composing the smoothed image, and a gain value g; and the gain value setting means set the gain value g based on input initial gain value g₀, reference gain value 1, and an attenuation value attn(Th₁, Th₂, L_(c)) calculated using a first luminance threshold value Th₁, a second luminance threshold value Th₂, and luminance L_(c) of pixels composing the tone-converted image.
 5. The image processing apparatus according to claim 1, characterized in that: the reduced image generation means generate a reduced image by converting the input image into the tone-converted image based on the conversion function and reducing a size of the tone-converted image; the correction information acquisition means acquire correction information including a slope of the conversion function; and the grayscale conversion means correct contrast of the tone-converted image based on the reduced image and the slope of the conversion function.
 6. (canceled)
 7. The image processing apparatus according to claim 5, characterized by further comprising: hold means for holding the reduced image generated by the reduced image generation means and the correction information acquired by the correction means; wherein the hold means holds the reduced image corresponding to a previous frame's image and a slope of the conversion function applied to the previous frame's image, and the grayscale conversion means corrects the contrast of the tone-converted image based on the reduced image of the previous frame and the slope of the conversion function, both stored in the hold means.
 8. An image processing method characterized by comprising: a reduced image generation step for generating a reduced image from an input image; a correction information acquisition step for acquiring a correction information of the input image based on the reduced image; and a grayscale conversion step for converting grayscale of the input image; wherein the grayscale conversion step corrects contrast of the input image using the correction information, as a processing to be performed before and/or after the grayscale is converted. 